We use the star formation history map of the Large Magellanic Cloud to study the sites of the eight smallest (and presumably youngest) supernova remnants in the Cloud: SN 1987A, N158A, N49, and N63A (core collapse remnants), 0509-67.5, 0519-69.0, N103B, and DEM L71 (Type Ia remnants). The local star formation histories provide unique insights into the nature of the supernova progenitors, which we compare with the properties of the supernova explosions derived from the remnants themselves and from supernova light echoes. We find that all the core collapse supernovae that we have studied are associated with vigorous star formation in the recent past. Stars more massive than 21.5Msun are very scarce around SNR N49, implying that the magnetar SGR 0526-66 in this SNR was either formed elsewhere or came from a progenitor with a mass well below the 30Msun threshold suggested in the literature. Three of our four Ia SNRs are associated with old, metal poor stellar populations. This includes SNR 0509-67.5, which is known to have been originated by an extremely bright Type Ia event, and yet is located very far away from any sites of recent star formation, in a population with a mean age of 7.9 Gyr. The Type Ia SNR N103B, on the other hand, is associated with recent star formation, and might have had a relatively younger and more massive progenitor with substantial mass loss before the explosion. We discuss these results in the context of our present understanding of core collapse and Type Ia supernova progenitors.