The effect of an electric field on the conductance of ultrathin films of metals deposited on substrates coated with a thin layer of amorphous Ge was investigated. A contribution to the conductance modulation symmetric with respect to the polarity of the applied electric field was found in regimes in which there was no sign of glassy behavior. For films with thicknesses that put them on the insulating side of the superconductor-insulator transition, the conductance increased with electric field, whereas for films that were becoming superconducting it decreased. Application of magnetic fields to the latter, which reduce the transition temperature and ultimately quench superconductivity, changed the sign of the reponse of the conductance to electric field back to that found for insulators. We propose that this symmetric response to capacitive charging is a consequence of changes in the conductance of the a-Ge layer, and is not a fundamental property of the physics of the superconductor-insulator transition as previously suggested.