The Rayleigh limit has so far applied to all microscopy techniques that rely on linear optical interaction and detection in the far field. Here we demonstrate that detecting the light emitted by an object in higher-order transverse electromagnetic modes (TEMs) can help achieving sub-Rayleigh precision for a variety of microscopy-related tasks. Using optical heterodyne detection in TEM01, we measure the position of coherently and incoherently emitting objects to within 0.0015 and 0.012 of the Rayleigh limit, respectively, and determine the distance between two incoherently emitting slits positioned within 0.28 of the Rayleigh limit with a precision of 0.019 of the Rayleigh limit. Extending our technique to higher-order TEMs enables full imaging with resolution significantly below the Rayleigh limit in a way that is reminiscent of quantum tomography of optical states.