Terrestrial planets orbiting M dwarfs may soon be observed with the James Webb Space Telescope (JWST) to characterize their atmospheric composition and search for signs of habitability or life. These planets may undergo significant atmospheric and ocean loss due to the superluminous pre-main-sequence phase of their host stars, which may leave behind abiotically-generated oxygen, a false positive for the detection of life. Determining if ocean loss has occurred will help assess potential habitability and whether or not any O2 detected is biogenic. In the solar system, differences in isotopic abundances have been used to infer the history of ocean loss and atmospheric escape (e.g. Venus, Mars). We find that isotopologue measurements using transit transmission spectra of terrestrial planets around late-type M dwarfs like TRAPPIST-1 may be possible with JWST, if the escape mechanisms and resulting isotopic fractionation were similar to Venus. We present analyses of post-ocean-loss O2- and CO2-dominated atmospheres, containing a range of trace gas abundances. Isotopologue bands are likely detectable throughout the near-infrared (1-8 um), especially 3-4 um, although not in CO2-dominated atmospheres. For Venus-like D/H ratios 100 times that of Earth, TRAPPIST-1 b transit signals of up to 79 ppm are possible by observing HDO. Similarly, 18O/16O ratios 100 times that of Earth produce signals at up to 94 ppm. Detection at S/N=5 may be attained on these bands with as few as four to eleven transits, with optimal use of JWST NIRSpec Prism. Consequently, H2O and CO2 isotopologues could be considered as indicators of past ocean loss and atmospheric escape for JWST observations of terrestrial planets around M dwarfs.