(Abridged) We study the link between quasars and the red galaxy population using a model for the self-regulated growth of supermassive black holes in mergers involving gas-rich galaxies. Using a model for quasar lifetimes and evolution motivated by hydrodynamical simulations of galaxy mergers, we de-convolve the observed quasar luminosity function at various redshifts to determine the rate of formation of black holes of a given final mass. Identifying quasar activity with the formation of spheroids in the framework of the merger hypothesis, this enables us to deduce the corresponding rate of formation of spheroids with given properties as a function of redshift. This allows us to predict, for the red galaxy population, the distribution of galaxy velocity dispersions, the mass function, mass density, star formation rates, the luminosity function in many observed wavebands (NUV, U, B, V, R, I, J, H, K), the total red galaxy number density and luminosity density, the distribution of colors as a function of magnitude and velocity dispersion for several different wavebands, the distribution of mass to light ratios vs. mass, the luminosity-size relations, and the typical ages and distribution of ages (formation redshifts) as a function of both mass and luminosity. For each of these quantities, we predict the evolution from redshift z=0-6. Each of our predictions agrees well with existing observations, without the addition of tunable parameters; the essential observational inputs come from the observed quasar luminosity function. These predictions are skewed by several orders of magnitude if we adopt simpler, traditional models of quasar lifetimes in which quasars turn on/off or follow simple exponential light curves, instead of the more complicated evolution implied by our simulations.