Ever since their discovery in the 1970s, UltraLuminous InfraRed Galaxies (ULIRGs; classically Lir>10^12Lsun) have fascinated astronomers with their immense luminosities, and frustrated them due to their singularly opaque nature, almost in equal measure. Over the last decade, however, comprehensive observations from the X-ray through to the radio have produced a consensus picture of local ULIRGs, showing that they are mergers between gas rich galaxies, where the interaction triggers some combination of dust-enshrouded starburst and AGN activity, with the starburst usually dominating. Very recent results have thrown ULIRGs even further to the fore. Originally they were thought of as little more than a local oddity, but the latest IR surveys have shown that ULIRGs are vastly more numerous at high redshift, and tantalizing suggestions of physical differences between high and low redshift ULIRGs hint at differences in their formation modes and local environment. In this review we look at recent progress on understanding the physics and evolution of local ULIRGs, the contribution of high redshift ULIRGs to the cosmic infrared background and the global history of star formation, and the role of ULIRGs as diagnostics of the formation of massive galaxies and large-scale structures.