We present the design of a highly compact High Field Scanning Probe Microscope (HF-SPM) for operation at cryogenic temperatures in an extremely high magnetic field, provided by a water-cooled Bitter magnet able to reach 38 T. The HF-SPM is 14 mm in diameter: an Attocube nano-positioner controls the coarse approach of a piezo resistive AFM cantilever to a scanned sample. The Bitter magnet constitutes an extreme environment for SPM due to the high level of vibrational noise; the Bitter magnet noise at frequencies up to 300 kHz is characterized and noise mitigation methods are described. The performance of the HF-SPM is demonstrated by topographic imaging and noise measurements at up to 30 T. Additionally, the use of the SPM as a three-dimensional dilatometer for magnetostriction measurements is demonstrated via measurements on a magnetically frustrated spinel sample.