A quantum dipole interacting with an optical cavity is one of the key models in cavity quantum electrodynamics (cavity-QED). To treat this system theoretically, the typical approach is to truncate the dipole to two levels. However, it has been shown that in the ultrastrong-coupling regime, this truncation naively destroys gauge invariance. By truncating in a manner consistent with the gauge principle, we introduce master equations to compute gauge-invariant emission spectra and quantum correlation functions which show significant disagreement with previous results obtained using the standard quantum Rabi model, with quantitative differences already present in the strong coupling regime. Explicit examples are shown using both the dipole gauge and the Coulomb gauge.