Observations of the accretion powered millisecond pulsar SAX J1808.4-3658 have revealed an interesting binary evolution, with the orbit of the system expanding at an accelerated rate. We use the recent finding that the accreted fuel in SAX J1808.4-3658 is hydrogen depleted to greatly refine models of the progenitor and prior evolution of the binary system. We constrain the initial mass of the companion star to 1.0-1.2 M$_{mathrm{odot}}$, more massive than previous evolutionary studies of this system have assumed. We also infer the system must have undergone strongly non-conservative mass transfer in order to explain the observed orbital period changes. Following Jia & Li (2015), we include mass loss due to the pulsar radiation pressure on the donor star, inducing an evaporative wind which is ejected at the inner Lagrangian point of the binary system. The resulting additional loss of angular momentum resolves the discrepancy between conservative mass transfer models and the observed orbital period derivative of this system. We also include a treatment of donor irradiation due to the accretion luminosity, and find this has a non-negligible effect on the evolution of the system.