Pyroelectric energy converter is a functional capacitor using pyroelectric material as the dielectric layer. Utilizing the first-order phase transformation of the material, the pyroelectric device can generate adequate electricity within small temperature fluctuations. However, most pyroelectric capacitors are leaking during energy conversion. In this paper, we analyze the thermodynamics of pyroelectric energy conversion with consideration of the electric leakage. Our thermodynamic model is verified by experiments using three phase-transforming ferroelectric materials with different pyroelectric properties and leakage behaviors. We demonstrate that the impact of leakage for electric generation is prominent, and sometimes may be confused with the actual power generation by pyroelectricity. We discover an ideal material candidate, (Ba,Ca)(Ti,Zr,Ce)O$_3$, which exhibits large pyroelectric current and extremely low leakage current. The pyroelectric converter made of this material generates 1.95 $mu$A/cm$^2$ pyroelectric current density and 0.2 J/cm$^3$ pyroelectric work density even after 1389 thermodynamic conversion cycles.