Weak measurement (WM) with state pre- and post-selection can amplify otherwise undetectable small signals and thus promise great potentials in precision measurements. Although frequency measurements offer the hitherto highest precision owing to stable narrow atomic transitions, it remains a long-standing interest to develop new schemes to further escalate their performance. Here, we propose and demonstrate a WM-enhanced spectroscopy technique which is capable of narrowing the resonance to 0.1 Hz in a room-temperature atomic vapor cell. Potential of this technique for precision measurement is demonstrated through weak magnetic field sensing. By judiciously pre- and post-selecting frequency-modulated input and output optical states in a nearly-orthogonal manner, a sensitivity of $text{7 fT/}sqrt{text{Hz}}$ near DC is achieved, using only one laser beam of $text{7 }text{mu W}$ power. Additionally, our results extend the WM framework to a non-Hermitian Hamiltonian, and shed new light in metrology and bio-magnetic field sensing applications.