The nature of the bipolar, $gamma$-ray Fermi bubbles (FB) is still unclear, in part because their faint, high-latitude X-ray counterpart has until now eluded a clear detection. We stack ROSAT data at varying distances from the FB edges, thus boosting the signal and identifying an expanding shell behind the southwest, southeast, and northwest edges, albeit not in the dusty northeast sector near Loop I. A Primakoff-like model for the underlying flow is invoked to show that the signals are consistent with halo gas heated by a strong, forward shock to $sim$keV temperatures. Assuming ion--electron thermal equilibrium then implies a $sim10^{56}$ erg event near the Galactic centre $sim7$ Myr ago. However, the reported high absorption-line velocities suggest a preferential shock-heating of ions, and thus more energetic ($sim 10^{57}$ erg), younger ($lesssim 3$ Myr) FBs.