We introduce a novel technique for empirically understanding galaxy evolution. We use empirically determined stellar evolution models to predict the past evolution of the Sloan Digital Sky Survey (SDSS-II) Luminous Red Galaxy (LRG) sample without any a-priori assumption about galaxy evolution. By carefully contrasting the evolution of the predicted and observed number and luminosity densities we test the passive evolution scenario for galaxies of different luminosity, and determine minimum merger rates. We find that the LRG population is not purely coeval, with some of galaxies targeted at z<0.23 and at z>0.34 showing different dynamical growth than galaxies targeted throughout the sample. Our results show that the LRG population is dynamically growing, and that this growth must be dominated by the faint end. For the most luminous galaxies, we find lower minimum merger rates than required by previous studies that assume passive stellar evolution, suggesting that some of the dynamical evolution measured previously was actually due to galaxies with non-passive stellar evolution being incorrectly modelled. Our methodology can be used to identify and match coeval populations of galaxies across cosmic times, over one or more surveys.