We present robust constraints on the stochastic gravitational waves (GWs) at Mpc scales from the cosmic microwave background (CMB) data. CMB constraints on GWs are usually characterized as the tensor-to-scalar ratio, assuming specifically a power-law form of the primordial spectrum, and are obtained from the angular spectra of CMB. Here, we relax the assumption of the power-law form, and consider to what extent one can constrain a monochromatic GW at shorter wavelengths. Previously, such a constraint has been derived at the wavelengths larger than the resolution scale of the CMB measurements, typically above $100$Mpc (below $10^{-16}$Hz in frequency). However, GWs whose wavelength is much shorter than $100$Mpc can imprint a small but non-negligible signal on CMB anisotropies at observed angular scales, $ell<1000$. Here, using the CMB temperature, polarization, and lensing data set, we obtain the best constraints to date at $10^{-16}-10^{-14}$Hz of the GWs produced before the time of decoupling, which are tighter than those derived from the astrometric measurements and upper bounds on extra radiations. In the future, the constraints on GWs at Mpc scales will be further improved by several orders of magnitude with the precision $B$-mode measurement on large scales, $ell<100$.