Raman scattering experiments on CdCr2S4 single crystals show pronounced anomalies in intensity and frequency of optical phonon modes with an onset temperature T*=130 K that coincides with the regime of giant magnetocapacitive effects. A loss of inversion symmetry and Cr off-centering are deduced from the observation of longitudinal optical and formerly infrared active modes for T<T_c=84 K. The intensity anomalies are attributed to the enhanced electronic polarizability of displacements that modulate the Cr-S distance and respective hybridization. Photo doping leads to an annihilation of the symmetry reduction. Our scenario of multiferroic effects is based on the near degeneracy of polar and nonpolar modes and the additional low energy scale due to hybridization.