We propose a new collider probe for axion-like particles (ALPs), and more generally for pseudo-Goldstone bosons: non-resonant searches which take advantage of the derivative nature of their interactions with Standard Model particles. ALPs can participate as off-shell mediators in the $s$-channel of $2 to 2$ scattering processes at colliders like the LHC. We exemplify the power of this novel type of search by deriving new limits on ALP couplings to gauge bosons via the processes $p p to Z Z$, $p p to gamma gamma$ and $p p to j j$ using Run 2 CMS public data, probing previously unexplored areas of the ALP parameter space. In addition, we propose future non-resonant searches involving the ALP coupling to other electroweak bosons and/or the Higgs particle.