We propose a new interpretation of the quasar luminosity function (LF), derived from physically motivated models of quasar lifetimes and light curves. In our picture, quasars evolve rapidly and their lifetime depends on both their instantaneous and peak luminosities. We study this model using simulations of galaxy mergers that successfully reproduce a wide range of observed quasar phenomena. With lifetimes inferred from the simulations, we deconvolve the observed quasar LF from the distribution of peak luminosities, and show that they differ qualitatively, unlike for the simple models of quasar lifetimes used previously. We find that the bright end of the LF traces the intrinsic peak quasar activity, but that the faint end consists of quasars which are either undergoing exponential growth to much larger masses and higher luminosities, or are in sub-Eddington quiescent states going into or coming out of a period of peak activity. The break in the LF corresponds directly to the maximum in the intrinsic distribution of peak luminosities, which falls off at both brighter and fainter luminosities. Our interpretation of the quasar LF provides a physical basis for the nature and slope of the faint-end distribution, as well as the location of the break luminosity.