We present an extensive numerical study of the time irreversibility of the dynamics of heavy inertial particles in three-dimensional, statistically homogeneous and isotropic turbulent flows. We show that the probability density function (PDF) of the increment, $W(tau)$, of a particles energy over a time-scale $tau$ is non-Gaussian, and skewed towards negative values. This implies that, on average, particles gain energy over a period of time that is longer than the duration over which they lose energy. We call this $textit{slow gain}$ and $textit{fast loss}$. We find that the third moment of $W(tau)$ scales as $tau^3$, for small values of $tau$. We show that the PDF of power-input $p$ is negatively skewed too; we use this skewness ${rm Ir}$ as a measure of the time-irreversibility and we demonstrate that it increases sharply with the Stokes number ${rm St}$, for small ${rm St}$; this increase slows down at ${rm St} simeq 1$. Furthermore, we obtain the PDFs of $t^+$ and $t^-$, the times over which $p$ has, respectively, positive or negative signs, i.e., the particle gains or loses energy. We obtain from these PDFs a direct and natural quantification of the the slow-gain and fast-loss of the particles, because these PDFs possess exponential tails, whence we infer the characteristic loss and gain times $t_{rm loss}$ and $t_{rm gain}$, respectively; and we obtain $t_{rm loss} < t_{rm gain}$, for all the cases we have considered. Finally, we show that the slow-gain in energy of the particles is equally likely in vortical or strain-dominated regions of the flow; in contrast, the fast-loss of energy occurs with greater probability in the latter than in the former.