The emerging empirical picture of galaxy stellar mass (Ms) assembly shows that galaxy population buildup proceeds from top to down in Ms. By connecting galaxies to LCDM halos and their histories, individual (average) Ms growth tracks can be inferred. These tracks show that massive galaxies assembled their Ms the earlier the more massive the halo, and that less massive galaxies are yet actively growing in Ms, the more active the less massive is the halo. The predicted star formation rates as a function of mass and the downsizing of the typical mass that separate active galaxies from the passive ones agree with direct observational determinations. This implies that the LCDM scenario is consistent with these observations. The challenge is now to understand the baryonic physics that drives the significant and systematical shift of the stellar mass assembly of galaxies from the mass assembly of their corresponding halos (from halo upsizing to galaxy downsizing).