We introduce the Making Galaxies in a Cosmological Context (MaGICC) program of smoothed particle hydrodynamics (SPH) simulations. We describe a parameter study of galaxy formation simulations of an L* galaxy that uses early stellar feedback combined with supernova feedback to match the stellar mass--halo mass relationship. While supernova feedback alone can reduce star formation enough to match the stellar mass--halo mass relationship, the galaxy forms too many stars before z=2 to match the evolution seen using abundance matching. Our early stellar feedback is purely thermal and thus operates like a UV ionization source as well as providing some additional pressure from the radiation of massive, young stars. The early feedback heats gas to >10^6 K before cooling to 10^4 K. The pressure from this hot gas creates a more extended disk and prevents more star formation prior to z=1 than supernovae feedback alone. The resulting disk galaxy has a flat rotation curve, an exponential surface brightness profile, and matches a wide range of disk scaling relationships. The disk forms from the inside-out with an increasing exponential scale length as the galaxy evolves. Overall, early stellar feedback helps to simulate galaxies that match observational results at low and high redshifts.