The field-effect-induced modulation of transport properties of 2-dimensional electron gases residing at the LaAlO$_3$/SrTiO$_3$ and LaGaO$_3$/SrTiO$_3$ interfaces has been investigated in a back-gate configuration. Both samples with crystalline and with amorphous overlayers have been considered. We show that the naive standard scenario, in which the back electrode and the 2-dimensional electron gas are simply modeled as capacitor plates, dramatically fails in describing the observed phenomenology. Anomalies appearing after the first low-temperature application of a positive gate bias, and causing a non-volatile perturbation of sample properties, are observed in all our samples. Such anomalies are shown to drive low-carrier density samples to a persistent insulating state. Recovery of the pristine metallic state can be either obtained by a long room-temperature field annealing, or, instantaneously, by a relatively modest dose of visible-range photons. Illumination causes a sudden collapse of the electron system back to the metallic ground state, with a resistivity drop exceeding four orders of magnitude. The data are discussed and interpreted on the base of the analogy with floating-gate MOSFET devices, which sheds a new light on the effects of back-gating on oxide-based 2-dimensional electron gases. A more formal approach, allowing for a semi-quantitative estimate of the relevant surface carrier densities for different samples and under different back-gate voltages, is proposed in the Appendix.