Recent experimental progress in condensed matter physics enables the observation of signatures of the parity anomaly in two-dimensional Dirac-like materials. Using effective field theories and analyzing band structures in external out-of-plane magnetic fields (orbital fields), we show that topological properties of quantum anomalous Hall (QAH) insulators are related to the parity anomaly. We demonstrate that the QAH phase survives in orbital fields, violates the Onsager relation, and can be therefore distinguished from a quantum Hall (QH) phase. As a fingerprint of the QAH phase in increasing orbital fields, we predict a transition from a quantized Hall plateau with $sigma_mathrm{xy}= -mathrm{e}^2/mathrm{h}$ to a not perfectly quantized plateau, caused by scattering processes between counterpropagating QH and QAH edge states. This transition can be especially important in paramagnetic QAH insulators, such as (Hg,Mn)Te/CdTe quantum wells, in which exchange interaction and orbital fields compete.