We present a definitive assessment of the role of Inverse Compton scattering of Cosmic Microwave Background photons (IC/CMB) in the context of radio galaxies. Owing to the steep increase of the CMB radiation energy density, IC/CMB is supposed to become progressively more important with respect to radio synchrotron cooling as the redshift increases. For typical energies at play, this process will up-scatter the CMB photons into the X-ray band, and is thus expected to yield a redshift-dependent, concurrent X-ray brightening and radio dimming of the jet-powered structures. Here we show how a conclusive proof of this effect hinges on high-resolution imaging data in which the extended lobes can be distinguished from the compact hot spots where synchrotron-self-Compton dominates the X-ray emission regardless of redshift. We analyze Chandra and Very Large Array data of 11 radio galaxies between 1.3<z<4.3, and demonstrate that the emission from their lobes is fully consistent with the expectations from IC/CMB in equipartition. Once the dependence on size and radio luminosity are properly accounted for, the measured lobe X-ray luminosities bear the characteristic (1+z)**4 proportionality expected of a CMB seed radiation field. Whereas this effect can effectively quench the (rest-frame) GHz radio emission from z>3 radio galaxies below <1 mJy, IC/CMB alone can not be responsible for a deficit in high-z, radio-loud AGN if--as we argue--such AGN typically have bright, compact hot spots.