In the last decade, detecting spin dynamics at the atomic scale has been enabled by combining techniques like electron spin resonance (ESR) or pump-probe spectroscopy with scanning tunneling microscopy (STM). Here, we demonstrate an ultra-high vacuum (UHV) STM operational at milliKelvin (mK) and in a vector magnetic field capable of both ESR and pump-probe spectroscopy. By implementing GHz compatible cabling, we achieve appreciable RF amplitudes at the junction while maintaining mK base temperature. We demonstrate the successful operation of our setup by utilizing two experimental ESR modes (frequency sweep and magnetic field sweep) on an individual TiH molecule on MgO/Ag(100) and extract the effective g-factor. We trace the ESR transitions down to MHz into an unprecedented low frequency band enabled by the mK base temperature. We also implement an all-electrical pump-probe scheme based on waveform sequencing suited for studying dynamics down to the nanoseconds range. We benchmark our system by detecting the spin relaxation time T1 of individual Fe atoms on MgO/Ag(100) and note a field strength and orientation dependent relaxation time.