Starting with the Gaudin-like Bethe ansatz equations associated with the quasi-exactly solved (QES) exceptional points of the asymmetric quantum Rabi model (AQRM) a spectral equivalence is established with QES hyperbolic Schrodinger potentials on the line. This leads to particular QES Poschl-Teller potentials. The complete spectral equivalence is then established between the AQRM and generalised Poschl-Teller potentials. This result extends a previous mapping between the symmetric quantum Rabi model and a QES Poschl-Teller potential. The complete spectral equivalence between the two systems suggests that the physics of the generalised Poschl-Teller potentials may also be explored in experimental realisations of the quantum Rabi model.