Cr$_2$O$_3$ is the archetypal magnetoelectric (ME) material, which has a linear coupling between electric and magnetic polarizations. Quadratic ME effects are forbidden for the magnetic point group of Cr$_2$O$_3$, due to space-time inversion symmetry. In Cr$_2$O$_3$ films grown by sputtering, we find a signature of a quadratic ME effect that is not found in bulk single crystals. We use Raman spectroscopy and magetization measurements to deduce the removal of space-time symmetry, and corroborate the emergence of the quadratic ME effect. We propose that meta-stable site-selective trace dopants remove the space, time, and space-time inversion symmetries from the original magnetic point group of bulk Cr$_2$O$_3$. We include the quadratic ME effect in a model describing the switching process during ME field cooling, and estimate the effective quadratic susceptibility value. The quadratic magnetoelectric effect in a uniaxial antiferromagnet is promising for multifunctional antiferromagnetic and magnetoelectric devices that can incorporate optical, strain-induced, and multiferroic effects.