We discover that, in the layered semiconductor Bi$_2$O$_2$Se, an incipient ferroelectric transition endows the material a surprisingly large dielectric permittivity, providing it with a robust protection against mobility degradation by extrinsic Coulomb scattering. Based on state-of-the-art first-principles calculations, we show that the low-temperature electron mobility of Bi$_2$O$_2$Se, taking into account both electron-phonon and ionized impurity scattering, can reach an unprecedented level of $10^5$ to $10^7$ cm$^2$V$^{-1}$s$^{-1}$ over a wide range of realistic doping levels. Moreover, a small elastic strain of 1.7% can drive Bi$_2$O$_2$Se toward the ferroelectric phase transition, which further induces a giant increase in the permittivity, enabling the strain-tuning of carrier mobility by orders of magnitude. These results open a new avenue for the discovery of high-mobility layered semiconductors via phase and dielectric engineering.