High-redshift binary quasars provide key insights into mergers and quasar activity, and are useful tools for probing the spatial kinematics and chemistry of galaxies along the line-of-sight. However, only three sub-10-kpc binaries have been confirmed above $z=1$. Gravitational lensing would provide a way to easily resolve such binaries, study them in higher resolution, and provide more sightlines, though the required alignment with a massive foreground galaxy is rare. Through image deconvolution of StanCam Nordic Optical Telescope (NOT) monitoring data, we reveal two further point sources in the known, $z approx 2.38$, quadruply lensed quasar (quad), J1721+8842. An ALFOSC/NOT long-slit spectrum shows that the brighter of these two sources is a quasar with $z = 2.369 pm 0.007$ based on the C III] line, while the C III] redshift of the quad is $z = 2.364 pm 0.003$. Lens modelling using point source positions rules out a single source model, favouring an isothermal lens mass profile with two quasar sources separated by $sim6.0$ kpc (0.73$^{prime prime}$) in projection. Given the resolving ability from lensing and current lensed quasar statistics, this discovery suggests a large population of undiscovered, unlensed sub-10-kpc binaries. We also analyse spectra of two images of the quad, showing narrow Ly$alpha$ emission within the trough of a proximate damped Ly$alpha$ absorber (PDLA). An apparent mismatch between the continuum and narrow line flux ratios provides a new potential tool for simultaneously studying microlensing and the quasar host galaxy. Signs of the PDLA are also seen in the second source, however a deeper spectrum is still required to confirm this. Thanks to the multiple lines-of-sight from lensing and two quasar sources, this system offers simultaneous sub-parsec and kpc-scale probes of a PDLA.