Cosmic photons can be efficiently collected by broadband intensity mapping but information on their emission redshift and frequency is largely lost. We introduce a technique to statistically recover these otherwise collapsed dimensions by exploiting information in spatial fluctuations and apply it to the Galaxy Evolution Explorer (GALEX) All Sky and Medium Imaging Surveys. By spatially cross-correlating photons in the GALEX far-UV (1500A) and near-UV (2300A) bands with a million spectroscopic objects in the Sloan Digital Sky Survey as a function of redshift, we robustly detect the redshift-dependent intensity of the UV background (UVB) modulated by its clustering bias up to $zsim2$. These measurements clearly reveal the imprints of UVB spectral features redshifting through the filters. Using a simple parameterization, we simultaneously fit a UVB emissivity and clustering bias factor to these observations and constrain the main spectral features of the UVB spectrum: (i) the Lyman break, (ii) the non-ionizing UV continuum, which agrees with the Haardt & Madau model but does not rely on any assumption regarding the nature of the sources, and (iii) the Ly$alpha$ emission, whose luminosity density is consistent with estimates of the combined galaxy and AGN contributions at $zsim1$. Because the technique probes the total background including low surface brightness emission, we place constraints on the amount of UV light originating from the diffuse intergalactic medium (IGM). Finally, the clustering bias of UV photons is found to be chromatic and evolving. Our frequency- and redshift-dependent UVB measurement delivers a summary statistic of the universes net radiation output from stars, black holes, and the IGM combined.