The Chern-Simons term, through which the cosmic Axion-like field couples to the electromagnetic field, has the effect to rotate CMB polarization directions and to break the CPT symmetry. This rotation will change the CMB power spectra, no matter isotropic or anisotropic the rotation angle is. In this paper we revisit this issue by further considering the correlations between the (anisotropic) rotation angle $alpha$ and the CMB temperature and (unrotated) $E$ polarization fields. These correlations could be generated in the Axion-like models with nonzero potential under the adiabatic initial condition. We first investigate how these correlations contribute further modifications to the CMB power spectra, then calculate the CMB bispectra for the temperature and rotated polarization fields. These bispectra would vanish if the $Talpha$ and $Ealpha$ correlations are absent. So, they are useful in searching for CPT violation and the $Talpha$ and $Ealpha$ correlations arisen in the Axion-like models.