We present the design and performance of a cryogenic scanning tunneling microscope (STM) which operates inside a water-cooled Bitter magnet, which can attain a magnetic field of up to 38 T. Due to the high vibration environment generated by the magnet cooling water, a uniquely designed STM and vibration damping system are required. The STM scan head is designed to be as compact and rigid as possible, to minimize the effect of vibrational noise as well as fit the size constraints of the Bitter magnet. The STM uses a differential screw mechanism for coarse tip - sample approach, and operates in helium exchange gas at cryogenic temperatures. The reliability and performance of the STM are demonstrated through topographic imaging and scanning tunneling spectroscopy (STS) on highly oriented pyrolytic graphite (HOPG) at T = 4.2 K and in magnetic fields up to 34 T.