We present robust statistical estimates of the accuracy of early-type galaxy stellar masses derived from spectral energy distribution (SED) fitting as functions of various empirical and theoretical assumptions. Using large samples consisting of 40,000 galaxies from the Sloan Digital Sky Survey, of which 5,000 are also in the UKIRT Infrared Deep Sky Survey, with spectroscopic redshifts in the range 0.05 leq z leq 0.095, we test the reliability of some commonly used stellar population models and extinction laws for computing stellar masses. Spectroscopic ages (t), metallicities (Z), and extinctions (A) are also computed from fits to SDSS spectra using various population models. These constraints are used in additional tests to estimate the systematic errors in the stellar masses derived from SED fitting, where t, Z, and A are typically left as free parameters. We find reasonable agreement in mass estimates among stellar population models, with variation of the IMF and extinction law yielding systematic biases on the mass of nearly a factor of 2, in agreement with other studies. Removing the near-infrared bands changes the statistical bias in mass by only 0.06 dex, adding uncertainties of 0.1 dex at the 95% CL. In contrast, we find that removing an ultraviolet band is more critical, introducing 2{sigma} uncertainties of 0.15 dex. Finally, we find that stellar masses are less affected by absence of metallicity and/or dust extinction knowledge. However, there is a definite systematic offset in the mass estimate when the stellar population age is unknown, up to a factor of 2.5 for very old (12 Gyr) stellar populations. We present the stellar masses for our sample, corrected for the measured systematic biases due to photometrically determined ages, finding that age errors produce lower stellar masses by 0.15 dex, with errors of 0.02 dex at the 95% CL for the median stellar age subsample.