We quantify the systematic effects on the stellar mass function which arise from assumptions about the stellar population, as well as how one fits the light profiles of the most luminous galaxies at z ~ 0.1. When comparing results from the literature, we are careful to separate out these effects. Our analysis shows that while systematics in the estimated comoving number density which arise from different treatments of the stellar population remain of order < 0.5 dex, systematics in photometry are now about 0.1 dex, despite recent claims in the literature. Compared to these more recent analyses, previous work based on Sloan Digital Sky Survey (SDSS) pipeline photometry leads to underestimates of rho_*(> M_*) by factors of 3-10 in the mass range 10^11 - 10^11.6 M_Sun, but up to a factor of 100 at higher stellar masses. This impacts studies which match massive galaxies to dark matter halos. Although systematics which arise from different treatments of the stellar population remain of order < 0.5 dex, our finding that systematics in photometry now amount to only about 0.1 dex in the stellar mass density is a significant improvement with respect to a decade ago. Our results highlight the importance of using the same stellar population and photometric models whenever low and high redshift samples are compared.