We report the temperature dependent mid- and near-infrared spectra of K4C60, Rb4C60 and Cs4C60. The splitting of the vibrational and electronic transitions indicates a molecular symmetry change of C604- which brings the fulleride anion from D2h to either a D3d or a D5d distortion. In contrast to Cs4C60, low temperature neutron diffraction measurements did not reveal a structural phase transition in either K4C60 and Rb4C60. This proves that the molecular transition is driven by the molecular Jahn-Teller effect, which overrides the distorting potential field of the surrounding cations at high temperature. In K4C60 and Rb4C60 we suggest a transition from a static to a dynamic Jahn-Teller state without changing the average structure. We studied the librations of these two fullerides by temperature dependent inelastic neutron scattering and conclude that both pseudorotation and jump reorientation are present in the dynamic Jahn-Teller state.