The energy resolution per bandwidth $E_R$ is a figure of merit that combines the field resolution, bandwidth or duration of the measurement, and size of the sensed region. Several different dc magnetometer technologies approach $E_R = hbar$, while to date none has surpassed this level. This suggests a technology-spanning quantum limit, a suggestion that is strengthened by model-based calculations for nitrogen-vacancy centres in diamond, for superconducting quantum interference device (SQUID) sensors, and for some optically-pumped alkali-vapor magnetometers, all of which predict a quantum limit close to $E_R = hbar$. Here we review what is known about energy resolution limits, with the aim to understand when and how $E_R$ is limited by quantum effects. We include a survey of reported sensitivity versus size of the sensed region for more than twenty magnetometer technologies, review the known model-based quantum limits, and critically assess possible sources for a technology-spanning limit, including zero-point fluctuations, magnetic self-interaction, and quantum speed limits. Finally, we describe sensing approaches that appear to be unconstrained by any of the known limits, and thus are candidates to surpass $E_R = hbar$.