Massive Population II galaxies undergoing the first phase of vigorous star formation after the initial Population III stage should have high energy densities and silicate-rich interstellar dust. We have modeled the resulting far-infrared spectral energy distributions (SEDs), demonstrating that they are shifted substantially to bluer (`warmer) wavelengths relative to the best fitting ones at z ~ 3, and with strong outputs in the 10 - 40 micron range. When combined with a low level of emission by carbon dust, their SEDs match that of Haro 11, a local moderately-low-metallicity galaxy undergoing a very young and vigorous starburst that is likely to approximate the relevant conditions in young Population II galaxies. We expect to see similar SEDs at high redshifts (z >= 5) given the youth of galaxies at this epoch. In fact, we find a progression with redshift in observed galaxy SEDs, from those resembling local ones at 2 < z < 4 to a closer resemblance with Haro 11 at 5 < z < 7. In addition to the insight on conditions in high redshift galaxies, this result implies that estimates of the total infrared luminosities at z ~ 6 based on measurements near lambda ~ 1 mm can vary by factors of 2 - 4, depending on the SED template used. Currently popular modified blackbodies or local templates can result in significant underestimates compared with the preferred template based on the SED of Haro 11.