Precision cosmology provides a sensitive probe of extremely weakly coupled states due to thermal freeze-in production, with subsequent decays impacting physics during well-tested cosmological epochs. We explore the cosmological implications of the freeze-in production of a new scalar $S$ via the super-renormalizable Higgs portal. If the mass of $S$ is at or below the electroweak scale, peak freeze-in production occurs during the electroweak epoch. We improve the calculation of the freeze-in abundance by including all relevant QCD and electroweak production channels. The resulting abundance and subsequent decay of $S$ is constrained by a combination of X-ray data, cosmic microwave background anisotropies and spectral distortions, $N_{rm eff}$, and the consistency of BBN with observations. These probes constrain technically natural couplings for such scalars from $m_S sim$ keV all the way to $m_S sim 100$ GeV. The ensuing constraints are similar in spirit to typical beam bump limits, but extend to much smaller couplings, down to mixing angles as small as $theta_{Sh} sim 10^{-16}$, and to masses all the way to the electroweak scale.