The dynamical Casimir effect (DCE) manifests itself in the ultrastrong matter-field coupling (USC) regime, as a consequence of the nonadiabatic change of some parameters of a system. We show that the DCE is a fundamental limitation for standard quantum protocols based on quantum Rabi oscillations, implying that new schemes are required to implement high-fidelity ultrafast quantum gates. Our results are illustrated by means of a paradigmatic quantum communication protocol, i.e., quantum state transfer.