Inelastic neutron scattering measurements were performed on single crystals of the antiferromagnetic compound Mn5Si3 in order to investigate the relation between the spin dynamics and the magneto-thermodynamic properties. It is shown that among the two stable antiferromagnetic phases of this compound, the high temperature one has an unusual magnetic excitation spectrum where propagative spin-waves and diffuse spin-fluctuations coexist. Moreover, it is evidenced that the inverse magnetocaloric effect of Mn5Si3, the cooling by adiabatic magnetization, is associated with field induced spin-fluctuations.