A model of Lorentz invariant random fluctuations in photon polarization is presented. The effects are frequency dependent and affect the polarization of photons as they propagate through space. We test for this effect by confronting the model with the latest measurements of polarization of Cosmic Microwave Background (CMB) photons.