We report unique EVLA observations of SN 2011fe representing the most sensitive radio study of a Type Ia supernova to date. Our data place direct constraints on the density of the surrounding medium at radii ~10^15-10^16 cm, implying an upper limit on the mass loss rate from the progenitor system of Mdot <~ 6 x 10^-10 Msol/yr (assuming a wind speed of 100 km/s), or expansion into a uniform medium with density n_CSM <~ 6 cm^-3. Drawing from the observed properties of non-conservative mass transfer among accreting white dwarfs, we use these limits on the density of the immediate environs to exclude a phase space of possible progenitors systems for SN 2011fe. We rule out a symbiotic progenitor system and also a system characterized by high accretion rate onto the white dwarf that is expected to give rise to optically-thick accretion winds. Assuming that a small fraction, 1%, of the mass accreted is lost from the progenitor system, we also eliminate much of the potential progenitor parameter space for white dwarfs hosting recurrent novae or undergoing stable nuclear burning. Therefore, we rule out the most popular single degenerate progenitor models for SN 2011fe, leaving a limited phase space inhabited by some double degenerate systems and exotic progenitor scenarios.