Yeah, for all we know he destroyed it before his death, or Voldemort took it from him after their paths crossed.