Current measurements of the $gamma$-ray Fermi bubbles (FB) are based on model-dependent tracers, carry substantial systematic uncertainties, and are at some tension with each other. We show that gradient filters pick out the FB edges, which are found to smoothly connect to the bipolar X-ray structure emanating from the Galactic center, thus supporting the interpretation of the FBs as a Galactic-scale phenomenon. The sharp edges facilitate a direct, model-free measurement of the peripheral FB spectrum. The result is strikingly similar to the full FB-integrated spectrum, softened by a power law of index $etasimeq (0.2mbox{--}0.3)$. This is naturally explained, in both hadronic and leptonic models, if cosmic rays are injected at the edge, and diffuse away preferentially at higher energies $E$. The inferred, averaged diffusion function in the (more plausible) leptonic model, $D(E)simeq 10^{29.5}(E/10mbox{ GeV})^{0.48pm0.02}mbox{ cm}^2mbox{ s}^{-1}$, is consistent with estimates for Kraichnan-like turbulence. Our results, in particular the minute spatial variations in $eta$, indicate that the FB edge is a strong, Mach $gtrsim5$, forward shock.