We study the effects of finite temperature on normal state properties of a metal near a quantum critical point to an antiferromagnetic or Ising-nematic state. At $T = 0$ bosonic and fermionic self-energies are traditionally computed within Eliashberg theory and obey scaling relations with characteristic power-laws. Quantum Monte Carlo (QMC) simulations have shown strong systematic deviations from these predictions, casting doubt on the validity of the theoretical analysis. We extend Eliashberg theory to finite $T$ and argue that for the $T$ range accessible in the QMC simulations, the scaling forms for both fermionic and bosonic self energies are quite different from those at $T = 0$. We compare finite $T$ results with QMC data and find good agreement for both systems. This, we argue, resolves the key apparent contradiction between the theory and the QMC simulations.