Off-resonant error for a driven quantum system refers to interactions due to the input drives having non-zero spectral overlap with unwanted system transitions. For the cross-resonance gate, this includes leakage as well as off-diagonal computational interactions that lead to bit-flip error on the control qubit. In this work, we quantify off-resonant error, with more focus on the less studied off-diagonal control interactions, for a direct CNOT gate implementation. Our results are based on numerical simulation of the dynamics, while we demonstrate the connection to time-dependent Schrieffer-Wolff and Magnus perturbation theories. We present two methods for suppressing such error terms. First, pulse parameters need to be optimized so that off-resonant transition frequencies coincide with the local minima due to the pulse spectrum sidebands. Second, we show the advantage of a $Y$-DRAG pulse on the control qubit in mitigating off-resonant error. Depending on qubit-qubit detuning, the proposed methods can improve the average off-resonant error from approximately $10^{-3}$ closer to the $10^{-4}$ level for a direct CNOT calibration.