The Berry phase (BP) in a quantized light field demonstrated more than a decade ago (Phys. Rev. Lett. 89, 220404) has attracted considerable attentions, since it plays an important role in the cavity quantum electrodynamics. However, it is argued in a recent paper ( Phys. Rev. Lett. 108, 033601) that such a BP is just due to the rotating wave approximation (RWA) and the relevant BP should vanish beyond this approximation. Based on a consistent analysis we conclude in this letter that the BP in a generic Rabi model actually exists, no matter whether the RWA is applied. The existence of BP is also generalized to a three-level atom in the quantized cavity field.