The strongest interaction between microscopic spins in magnetic materials is the exchange interaction $J_text{ex}$. Therefore, ultrafast control of $J_text{ex}$ holds the promise to control spins on ultimately fast timescales. We demonstrate that time-periodic modulation of the electronic structure by electric fields can be used to reversibly control $J_text{ex}$ on ultrafast timescales in extended antiferromagnetic Mott insulators. In the regime of weak driving strength, we find that $J_text{ex}$ can be enhanced and reduced for frequencies below and above the Mott gap, respectively. Moreover, for strong driving strength, even the sign of $J_text{ex}$ can be reversed and we show that this causes time reversal of the associated quantum spin dynamics. These results suggest wide applications, not only to control magnetism in condensed matter systems, for example, via the excitation of spin resonances, but also to assess fundamental questions concerning the reversibility of the quantum many-body dynamics in cold atom systems.