On August 24 (UT) the Palomar Transient Factory (PTF) discovered PTF11kly (SN 2011fe), the youngest and most nearby type Ia supernova (SN Ia) in decades. We followed this event up in the radio (centimeter and millimeter bands) and X-ray bands, starting about a day after the estimated explosion time. We present our analysis of the radio and X-ray observations, yielding the tightest constraints yet placed on the pre-explosion mass-loss rate from the progenitor system of this supernova. We find a robust limit of dM/dt<10^-8 (w/100 km/s) [M_solar/yr] from sensitive X-ray non-detections, as well as a similar limit from radio data, which depends, however, on assumptions about microphysical parameters. We discuss our results in the context of single-degenerate models for SNe Ia and find that our observations modestly disfavor symbiotic progenitor models involving a red giant donor, but cannot constrain systems accreting from main-sequence or sub-giant stars, including the popular supersoft channel. In view of the proximity of PTF11kly and the sensitivity of our prompt observations we would have to wait for a long time (decade or longer) in order to more meaningfully probe the circumstellar matter of Ia supernovae.