The North Polar Spur (NPS) is one of the largest structures observed in the Milky Way in both the radio and soft x-rays. While several predictions have been made regarding the origin of the NPS, modelling the structure is difficult without precise distance constraints. In this paper, we determine accurate distances to the southern terminus of the NPS and toward latitudes ranging up to 55$^{circ}$. First, we fit for the distance and extinction to stars toward the NPS using optical and near-infrared photometry and Gaia DR2 astrometry. We model these per-star distance-extinction estimates as being caused by dust screens at unknown distances, which we fit for using a nested sampling algorithm. We then compare the extinction to the Spur derived from our 3D dust modelling with integrated independent measures from XMM-Newton X-ray absorption and HI column density measures. We find that we can account for nearly 100% of the total column density of the NPS as lying within 140 pc for latitudes $>26^{circ}$ and within 700 pc for latitudes $< 11^{circ}$. Based on the results, we conclude that the NPS is not associated with the Galactic Centre or the Fermi bubbles. Instead, it is likely associated, especially at higher latitudes, with the Sco-Cen association.