Dissipative quantum Rabi System, a finite-component system composed of a single two-level atom interacting with an optical cavity field mode, exhibits a quantum phase transition, which can be exploited to greatly enhance the estimation precision of unitary parameters (frequency and coupling strength). Here, using the quantum Langevin equation, standard mean field theory and adiabatic elimination, we investigate the quantum thermometry of a thermal bath surrounding the atom with quantum optical probes. With the increase of coupling strength between the atom and the cavity field, two kinds of singularities can be observed. One type of singularity is the exceptional point (EP) in the anti-parity-time (anti-$mathcal{PT}$) symmetrical cavity field. The other type of singularity is the critical point (CP) of phase transition from the normal to superradiant phase. We show that the optimal measurement precision occurs at the CP, instead of the EP. And the direct photon detection represents an excellent proxy for the optimal measurement near the CP. In the case where the thermal bath to be tested is independent of the extra thermal bath interacting with the cavity field, the estimation precision of the temperature always increases with the coupling strength. Oppositely, if the thermal bath to be tested is in equilibrium with the extra bath interacting with the cavity field, noises that suppress the information of the temperature will be introduced when increasing the coupling strength unless it is close to the CP.