Optical spectroscopy is a fundamental tool in numerous areas of science and technology. Much effort has focused on miniaturizing spectrometers, but thus far at the cost of high spectral resolution and broad operating range. Here, we describe a compact spectrometer without this trade-off. The device relies on imaging multi-mode interference from leaky modes along a highly multimode tapered optical fiber, resulting in spectrally distinguishable images that form a basis for reconstructing an incident light spectrum. This tapered fiber multimode interference spectrometer enables the acquisition of broadband spectra in a single camera exposure with a measured resolution of 40 pm in the visible spectrum and 10 pm in the infrared spectrum, which are comparable to the performance of grating spectrometers. Spectroscopy from 500 nm to 1600 nm is demonstrated, though operation across the entire transparency window of silica fibers is possible. Multimode interference spectroscopy of leaky modes is suitable in a variety of device geometries, including planar waveguides in a broad range of transparent materials. We anticipate that this technique will greatly expand the availability of high-performance spectroscopy in a wide range of applications.