Deep surveys of the sky at millimeter wavelengths have revealed a population of ultra-luminous infrared galaxies (ULIRGs) at high redshifts. These appear similar to local objects of similar luminosities (such as Arp220) but are much more ``important at high redshift than at low reshift, in the sense that they represent a much larger fraction of the total luminous output of the distant Universe than they do locally. In fact the ULIRGs at high redshift are producing a significant fraction (>= 15%) of the total luminous output of the Universe averaged over all wavelengths and all epochs. The high z ULIRGs could plausibly be responsible for producing the metal-rich spheroidal components of galaxies, including the bulges that are the subject of this conference. In this case we would infer from the redshift distribution of the sources that much of this activity is probably happening relatively recently at z <= 2.