Measurements of ferroelectric polarization and dielectric constant were done on $R$Mn$_2$O$_5$ ($R$=Tb, Dy, and Ho) with applied hydrostatic pressures of up to 18 kbar. At ambient pressure, distinctive anomalies were observed in the temperature profile of both physical properties at critical temperatures marking the onset of long range AFM order (T$_{N1}$), ferroelectricity (T$_{C1}$) as well as at temperatures when anomalous changes in the polarization, dielectric constant and spin wave commensurability have been previously reported. In particular, the step in the dielectric constant at low temperatures (T$_{C2}$), associated with both a drop in the ferroelectric polarization and an incommensurate magnetic structure, was shown to be suddenly quenched upon passing an $R$-dependent critical pressure. This was shown to correlate with the stabilization of the high ferroelectric polarization state which is coincident with the commensurate magnetic structure. The observation is suggested to be due to a pressure induced phase transition into a commensurate magnetic structure as exemplified by the pressure-temperature ($p$-$T$) phase diagrams constructed in this work. The $p$-$T$ phase diagrams are determined for all three compounds.