We present EzGal, a flexible python program designed to easily generate observable parameters (magnitudes, colors, mass-to-light ratios) for any stellar population synthesis (SPS) model. As has been demonstrated by various authors, the choice of input SPS models can be a significant source of systematic uncertainty. A key strength of EzGal is that it enables simple, direct comparison of different models sets. EzGal is also capable of generating composite stellar population models (CSPs) and can interpolate between metallicities for a given model set. We have created a web interface to run EzGal and generate observables for a variety of star formation histories and model sets. We make many commonly used SPS models available from this interface; the BC03 models, an updated version of these models, the Maraston models, the BaSTI models, and finally the FSPS models. We use EzGal to compare magnitude predictions for the model sets as a function of wavelength, age, metallicity, and star formation history. We recover the well-known result that the models agree best in the optical for old, solar metallicity models, with differences at the ~0.1 magnitude level. The most problematic regime for SPS modeling is for young ages (<2 Gyrs) and long wavelengths (lambda >7500 Angstroms) where scatter between models can vary from 0.3 mags (Sloan i) to 0.7 mags (Ks). We find that these differences are best understood as general uncertainties in SPS modeling. Finally we explore a more physically motivated example by generating CSPs with a star formation history matching the global star formation history of the universe. We demonstrate that the wavelength and age dependence of SPS model uncertainty translates into a redshift dependent model uncertainty, highlighting the importance of a quantitative understanding of model differences when comparing observations to models as a function of redshift.