Using first-principles calculations and group-theoretical methods, we study the origin and stabilization of ferrielectricity (FiE) in CuInP$_2$Se$_6$. We find that the polar distortions of the metal atoms create most of the polarization in the FiE phase. Surprisingly, the stabilization of the FiE phase comes from an anharmonic coupling between the polar mode and a fully symmetric Raman-active mode comprising primarily of the Se atoms. This coupling is large even down to the monolayer limit, and the degree of anharmonicity is comparable to improper ferroelectrics. Our results open up possibilities for dynamical control of the single-step ferroelectric switching barrier by tuning the Raman-active mode. These findings have important implications not only for designing next-generation microelectronic devices that can overcome the voltage-time dilemma but also in explaining the unconventional responses observed in CuInP$_2$Se$_6$ and similar layered thiophosphates.