Semiconductor holes with strong spin-orbit coupling allow all-electrical spin control, with broad applications ranging from spintronics to quantum computation. Using a two-dimensional hole system in a GaAs quantum well, we demonstrate a new mechanism of electrically controlling the Zeeman splitting, which is achieved through altering the hole wave vector $k$. We find a threefold enhancement of the in-plane $g-$factor $g_{parallel}(k)$. We introduce a new method for quantifying the Zeeman splitting from magnetoresistance measurements, since the conventional tilted field approach fails for two-dimensional systems with strong spin-orbit coupling. Finally, we show that the Rashba spin-orbit interaction suppresses the in-plane Zeeman interaction at low magnetic fields. The ability to control the Zeeman splitting with electric fields opens up new possibilities for future quantum spin-based devices, manipulating non-Abelian geometric phases, and realising Majorana systems in $p-$type superconductor systems.