In this work, we study the BCS-BEC crossover and quantum phase transition in a Fermi gas under Rashba spin-orbit coupling close to a Feshbach resonance. By adopting a two-channel model, we take into account of the closed channel molecules, and show t
hat combined with spin-orbit coupling, a finite background scattering in the open channel can lead to two branches of solution for both the two-body and the many-body ground states. The branching of the two-body bound state solution originates from the avoided crossing between bound states in the open and the closed channels, respectively. For the many-body states, we identify a quantum phase transition in the upper branch regardless of the sign of the background scattering length, which is in clear contrast to the case without spin-orbit coupling. For systems with negative background scattering length in particular, we show that the bound state in the open channel, and hence the quantum phase transition in the upper branch, are induced by spin-orbit coupling. We then characterize the critical detuning of the quantum phase transition for both positive and negative background scattering lengths, and demonstrate the optimal parameters for the critical point to be probed experimentally.
