The Fermi Gamma-ray Space Telescope has revealed a diffuse $gamma$-ray background at energies from 0.1 GeV to 1 TeV, which can be separated into Galactic emission and an isotropic, extragalactic component. Previous efforts to understand the latter have been hampered by the lack of physical models capable of predicting the $gamma$-ray emission produced by the many candidate sources, primarily active galactic nuclei and star-forming galaxies, leaving their contributions poorly constrained. Here we present a calculation of the contribution of star-forming galaxies to the $gamma$-ray background that does not rely on empirical scalings, and is instead based on a physical model for the $gamma$-ray emission produced when cosmic rays accelerated in supernova remnants interact with the interstellar medium. After validating the model against local observations, we apply it to the observed cosmological star-forming galaxy population and recover an excellent match to both the total intensity and the spectral slope of the $gamma$-ray background, demonstrating that star-forming galaxies alone can explain the full diffuse, isotropic $gamma$-ray background.