There have been recently several works studying the regularized stochastic heat equation (SHE) and Kardar-Parisi-Zhang (KPZ) equation in dimension $dgeq 3$ as the smoothing parameter is switched off, but most of the results did not hold in the full temperature regions where they should. Inspired by martingale techniques coming from the directed polymers literature, we first extend the law of large numbers for SHE obtained in [MSZ16] to the full weak disorder region of the associated polymer model and to more general initial conditions. We further extend the Edwards-Wilkinson regime of the SHE and KPZ equation studied in [GRZ18,MU17,DGRZ20] to the full $L^2$-region, along with multidimensional convergence and general initial conditions for the KPZ equation (and SHE), which were not proven before. To do so, we rely on a martingale CLT combined with a refinement of the local limit theorem for polymers.