Massive galaxies at high-z have smaller effective radii than those today, but similar central densities. Their size growth therefore relates primarily to the evolving abundance of low-density material. Various models have been proposed to explain this evolution, which have different implications for galaxy, star, and BH formation. We compile observations of spheroid properties as a function of redshift and use them to test proposed models. Evolution in progenitor gas-richness with redshift gives rise to initial formation of smaller spheroids at high-z. These systems can then evolve in apparent or physical size via several channels: (1) equal-density dry mergers, (2) later major or minor dry mergers with less-dense galaxies, (3) adiabatic expansion, (4) evolution in stellar populations & mass-to-light-ratio gradients, (5) age-dependent bias in stellar mass estimators, (6) observational fitting/selection effects. If any one of these is tuned to explain observed size evolution, they make distinct predictions for evolution in other galaxy properties. Only model (2) is consistent with observations as a dominant effect. It is the only model which allows for an increase in M_BH/M_bulge with redshift. Still, the amount of merging needed is larger than that observed or predicted. We therefore compare cosmologically motivated simulations, in which all these effects occur, & show they are consistent with all the observational constraints. Effect (2), which builds up an extended low-density envelope, does dominate the evolution, but effects 1,3,4, & 6 each contribute ~20% to the size evolution (a net factor ~2). This naturally also predicts evolution in M_BH-sigma similar to that observed.
Download