Observations of local star forming galaxies have revealed a correlation between the rate at which galaxies form stars and their X-Ray luminosity. We combine this correlation with the most recent observational constraints on the integrated star formation rate density, and find that star forming galaxies account for 5-20% of the total soft and hard X-ray backgrounds, where the precise number depends on the energy band and the assumed average X-ray spectral energy distribution of the galaxies below ~20 keV. If we combine the L_X-SFR relation with recently derived star formation rate function, then we find that star forming galaxies whose X-ray flux falls well (more than a factor of 10) below the detection thresholds of the Chandra Deep Fields, can fully account for the unresolved soft X-ray background, which corresponds to ~6% of its total. Motivated by this result, we put limits on the allowed redshift evolution of the parameter c_X equiv L_X/SFR, and/or its evolution towards lower and higher star formation rates. If we parametrize the redshift evolution of c_X ~ (1+z)^b, then we find that b leq 1.3 (95% CL). On the other hand, the observed X-ray luminosity functions (XLFs) of star forming galaxies indicate that c_X may be increasing towards higher redshifts and/or higher star formation rates at levels that are consistent with the X-ray background, but possibly at odds with the locally observed L_X-SFR relation.