To explore the connection between the global physical properties of galaxies and their far-infrared (FIR) spectral energy distributions (SEDs), we study the variation in the FIR SEDs of a set of hydrodynamically simulated galaxies that are generated by performing dust radiative transfer in post-processing. Our sample includes both isolated and merging systems at various stages of the merging process and covers infrared (IR) luminosities and dust masses that are representative of both low- and high-redshift galaxies. We study the FIR SEDs using principle component analysis (PCA) and find that 97% of the variance in the sample can be explained by two principle components (PCs). The first PC characterizes the wavelength of the peak of the FIR SED, and the second encodes the breadth of the SED. We find that the coefficients of both PCs can be predicted well using a double power law in terms of the IR luminosity and dust mass, which suggests that these two physical properties are the primary determinants of galaxies FIR SED shapes. Incorporating galaxy sizes does not significantly improve our ability to predict the FIR SEDs. Our results suggest that the observed redshift evolution in the effective dust temperature at fixed IR luminosity is not driven by geometry: the SEDs of $z sim 2-3$ ultraluminous IR galaxies (ULIRGs) are cooler than those of local ULIRGs not because the high-redshift galaxies are more extended but rather because they have higher dust masses at fixed IR luminosity. Finally, based on our simulations, we introduce a two-parameter set of SED templates that depend on both IR luminosity and dust mass.