We present a computation, within weakly-coupled thermal QCD, of the production rate of low invariant mass ($M^2 sim g^2 T^2$) dileptons, at next-to-leading order (NLO) in the coupling (which is $O(g^3 e^2 T^2)$). This involves extending the NLO calculation of the photon rate which we recently presented to the case of small nonzero photon invariant mass. Numerical results are discussed and tabulated forms and code are provided for inclusion in hydrodynamical models. We find that NLO corrections can increase the dilepton rate by up to 30-40% relative to leading order. We find that the electromagnetic response of the plasma for real photons and for small invariant mass but high energy dilepton pairs (e.g., $M^2 < (300:mathrm{MeV})^2$ but $p_T > 1 : mathrm{GeV}$) are close enough that dilepton pair measurements really can serve as Ersatz photon measurements. We also present a matching a la Ghisoiu and Laine between our results and results at larger invariant masses.