We study dynamical systems which admit action-angle variables at leading order which are subject to nearly resonant perturbations. If the frequencies characterizing the unperturbed system are not in resonance, the long-term dynamical evolution may be integrated by orbit-averaging over the high-frequency angles, thereby evolving the orbit-averaged effect of the perturbations. It is well known that such integrators may be constructed via a canonical transformation, which eliminates the high frequency variables from the orbit-averaged quantities. An example of this algorithm in celestial mechanics is the von Zeipel transformation. However if the perturbations are inside or close to a resonance, i.e. the frequencies of the unperturbed system are commensurate, these canonical transformations are subject to divergences. We introduce a canonical transformation which eliminates the high frequency phase variables in the Hamiltonian without encountering divergences. This leads to a well-behaved symplectic integrator. We demonstrate the algorithm through two examples: a resonantly perturbed harmonic oscillator and the gravitational three-body problem in mean motion resonance.