The study of comets affords a unique window into the birth, infancy, and subsequent history of the solar system. There is strong evidence that comets incorporated pristine interstellar material as well as processed nebular matter, providing insights into the composition and prevailing conditions over wide swaths of the solar nebula at the time of planet formation. Dynamically new Oort cloud comets harbor primitive ices that have been stored thousands of astronomical units from the Sun and have suffered minimal thermal or radiative processing since their emplacement ~4.5 Gyr ago. Periodic, more dynamically evolved comets such as the Halley-type and Jupiter-family comets reveal the effects of lives spent over a range of heliocentric distances, including perihelion passages into the very inner solar system. Systematically characterizing the information imprinted in the native ice compositions of these objects is critical to understanding the formation and evolution of the solar system, the presence of organic matter and water on the terrestrial planets, the chemistry present in protoplanetary disks around other stars, and the nature of interstellar interlopers such as 2I/Borisov. Although comet rendezvous and sample return missions can provide remarkable insights into the properties of a few short-period comets, the on-sky capacity necessary to perform population-level comet studies while simultaneously remaining sensitive to the paradigm-challenging science that individual comets can reveal can only be provided by remote sensing observations. Here we report the state-of-the-art in ground- and space-based remote sensing of cometary volatiles, review the remarkable progress of the previous decade, articulate the pressing questions that ground- and space-based work will address over the next ten years, and advocate for the technology and resources necessary to realize these aspirations.