Voltage control of ferromagnetism on the nanometer scale is highly appealing for the development of novel electronic devices. Here a key challenge is to implement and combine low power consumption, high operation speed, reliable reversibility and compatibility with semiconductor technology. Hybrid structures based on the assembly of ferromagnetic and semiconducting building blocks are attractive candidates in that respect as such systems bring together the properties of the isolated constituents: They are expected to show magnetic order as a ferromagnet and to be electrically tunable as a semiconductor. Here we demonstrate the electrical control of the exchange coupling in a hybrid consisting of a ferromagnetic Co layer and a semiconductor CdTe quantum well, separated by a thin non-magnetic (Cd,Mg)Te barrier. The effective magnetic field of the exchange interaction reaches up to 2.5 Tesla and can be turned on and off by application of 1 V bias across the heterostructure. The mechanism of this electric field control is essentially different from the conventional concept, in which wavefunctions are spatially redistributed to vary the exchange interaction, requiring high field strengths. Here we address instead control of the novel exchange mechanism that is mediated by elliptically polarized phonons emitted from the ferromagnet, i.e. the phononic ac Stark effect. An essential parameter of this coupling is the splitting between heavy and light hole states in the quantum well which can be varied by the electric field induced band bending. Thereby the splitting can be tuned with respect to the magnon-phonon resonance energy in the ferromagnet, leading to maximum coupling for flat band conditions. Our results demonstrate the feasibility of electrically controlled exchange coupling in hybrid semiconductor nanostructures at quite moderate electric field strengths.