We present an application of the da Cunha, Charlot & Elbaz (2008) model of the spectral energy distribution (SEDs) of galaxies from the ultraviolet to far-infrared to a small pilot sample of purely star-forming Ultra-Luminous Infrared Galaxies (ULIRGs). We interpret the observed SEDs of 16 ULIRGs using this physically-motivated model which accounts for the emission of stellar populations from the ultraviolet to the near-infrared and for the attenuation by dust in two components: an optically-thick starburst component and the diffuse ISM. The infrared emission is computed by assuming that all the energy absorbed by dust in these components is re-radiated at mid- and far-infrared wavelengths. This model allows us to derive statistically physical properties including star formation rates, stellar masses, as well as temperatures and masses of different dust components and plausible star formation histories. We find that, although the ultraviolet to near-infrared emission represents only a small fraction of the total power radiated by ULIRGs, observations in this wavelength range are important to understand the properties of the stellar populations and dust attenuation in the diffuse ISM of these galaxies. Furthermore, our analysis indicates that the use of mid-infrared spectroscopy from the Infrared Spectrograph on the Spitzer Space Telescope is crucial to obtain realistic estimates of the extinction to the central energy source, mainly via the depth of the 9.7-micron silicate feature, and thus accurately constrain the total energy balance. Our findings are consistent with the notion that, in the local Universe, the physical properties of ULIRGs are fundamentally different from those of galaxies with lower infrared luminosities and that local ULIRGs are the result of merger-induced starbursts. [abridged]