We report the discovery of two-phase unconventional superconductivity in CeRh$_2$As$_2$. Using thermodynamic probes, we establish that the superconducting critical field of its high-field phase is as high as 14 T, remarkable in a material whose transition temperature is 0.26 K. Furthermore, a $c$-axis field drives a transition between two different superconducting phases. In spite of the fact that CeRh$_2$As$_2$ is globally centrosymmetric, we show that local inversion-symmetry breaking at the Ce sites enables Rashba spin-orbit coupling to play a key role in the underlying physics. More detailed analysis identifies the transition from the low- to high-field states to be associated with one between states of even and odd parity.