Unconventional ferroelectricity, robust at reduced nanoscale sizes, exhibited by hafnia-based thin-films presents tremendous opportunities in nanoelectronics. However, the exact nature of polarization switching remains controversial. Here, we investigate epitaxial Hf0.5Zr0.5O2 (HZO) capacitors, interfaced with oxygen conducting metals (La0.67Sr0.33MnO3, LSMO) as electrodes, using atomic resolution electron microscopy while in situ electrical biasing. By direct oxygen imaging, we observe reversible oxygen vacancy migration from the bottom to the top electrode through HZO and reveal associated reversible structural phase transitions in the epitaxial LSMO and HZO layers. We follow the phase transition pathways at the atomic scale and identify that these mechanisms are at play both in tunnel junctions and ferroelectric capacitors switched with sub-millisecond pulses. Our results unmistakably demonstrate that oxygen voltammetry and polarization switching are intertwined in these materials.