Rydberg atom-based electrometry enables traceable electric field measurements with high sensitivity over a large frequency range, from gigahertz to terahertz. Such measurements are particularly useful for the calibration of radio frequency and terahertz devices, as well as other applications like near field imaging of electric fields. We utilize frequency modulated spectroscopy with active control of residual amplitude modulation to improve the signal to noise ratio of the optical readout of Rydberg atom-based radio frequency electrometry. Matched filtering of the signal is also implemented. Although we have reached similarly, high sensitivity with other read-out methods, frequency modulated spectroscopy is advantageous because it is well-suited for building a compact, portable sensor. In the current experiment, $sim 3 mu V cm^{-1}Hz^{-1/2}$ sensitivity is achieved and is found to be photon shot noise limited.