The outer solar system provides a unique, quiet vantage point from which to observe the universe around us, where measurements could enable several niche astrophysical science cases that are too difficult to perform near Earth. NASAs New Horizons mission comprises an instrument package that provides imaging capability from ultraviolet (UV) to near-infrared (near-IR) wavelengths with moderate spectral resolution located beyond the orbit of Pluto. A carefully designed survey with New Horizons can optimize the use of expendable propellant and the limited data telemetry bandwidth to allow several measurements, including a detailed understanding of the cosmic extragalactic background light; studies of the local and extragalactic UV background; measurements of the properties of dust and ice in the outer solar system; confirmation and characterization of transiting exoplanets; determinations of the mass of dark objects using gravitational microlensing; and rapid follow-up of transient events. New Horizons is currently in an extended mission designed to focus on Kuiper Belt science that will conclude in 2021. The astrophysics community has a unique, generational opportunity to use this mission for astronomical observation at heliocentric distances beyond 50 au in the next decade. In this paper, we discuss the potential science cases for such an extended mission, and provide an initial assessment of the most important operational requirements and observation strategies it would require. We conclude that New Horizons is capable of transformative science, and that it would make a valuable and unique asset for astrophysical science that is unlikely to be replicated in the near future.