We study systems of fully polarized ultracold atomic gases obeying Fermi statistics. The atomic transition interacts dispersively with a mode of a standing-wave cavity, which is coherently pumped by a laser. In this setup, the intensity of the intrac
avity field is determined by the refractive index of the atomic medium, and thus by the atomic density distribution. Vice versa, the density distribution of the atom is determined by the cavity field potential, whose depth is proportional to the intracavity field amplitude. In this work we show that this nonlinearity leads to an instability in the intracavity intensity that differs substantially from dispersive optical bistability, as this effect is already present in the regime, where the atomic dipole is proportional to the cavity field. Such instability is driven by the matter waves fluctuations and exhibits a peculiar dependence on the fluctuations in the atomic density distribution.
We investigate a paradigm example of cavity quantum electrodynamics with many body systems: an ultracold atomic gas inside a pumped optical resonator. In particular, we study the stability of atomic insulator-like states, confined by the mechanical p
otential emerging from the cavity field spatial mode structure. As in open space, when the optical potential is sufficiently deep, the atomic gas is in the Mott-like state. Inside the cavity, however, the potential depends on the atomic distribution, which determines the refractive index of the medium, thus altering the intracavity field amplitude. We derive the effective Bose-Hubbard model describing the physics of the system in one dimension and study the crossover between the superfluid -- Mott insulator quantum states. We determine the regions of parameters where the atomic insulator states are stable, and predict the existence of overlapping stability regions corresponding to competing insulator-like states. Bistable behavior, controlled by the pump intensity, is encountered in the vicinity of the shifted cavity resonance.
