We compute the spectral distortions of the Cosmic Microwave Background (CMB) polarization induced by non-linear effects in the Compton interactions between CMB photons and cold intergalactic electrons. This signal is of the $y$-type and is dominated by contributions arising from the reionized era. We stress that it is not shadowed by the thermal SZ effect which has no equivalent for polarization. We decompose its angular dependence into $E$- and $B$-modes, and we calculate the corresponding power spectra, both exactly and using a suitable Limber approximation that allows a simpler numerical evaluation. We find that $B$-modes are of the same order of magnitude as $E$-modes. Both spectra are relatively flat, peaking around $ell=280$, and their overall amplitude is directly related to the optical depth to reionization. Moreover, we find this effect to be one order of magnitude larger than the non-linear kinetic Sunyaev-Zeldovich effect in galaxy clusters. Finally, we discuss how to improve the detectability of our signal by cross-correlating it with other quantities sourced by the flow of intergalactic electrons.