The rotation of polarization occurs for light interacting with chiral materials. It requires the light states with opposite chiralities interact differently with the materials. We demonstrate analogous rotation of polarization also exists for chiral fermions interacting with quantum electrodynamics plasma with vorticity using chiral kinetic theory. We find that the rotation of polarization is perpendicular both to vorticity and fermion momentum. The effect also exists for chiral fermions in quantum chromodynamics plasma with vorticity. It could lead to generation of a vector current when the probe fermions contain momentum anisotropy.