Stardust grains recovered from meteorites provide high-precision snapshots of the isotopic composition of the stellar environment in which they formed. Attributing their origin to specific types of stars, however, often proves difficult. Intermediate-mass stars of 4-8 solar masses are expected to contribute a large fraction of meteoritic stardust. However, no grains have been found with characteristic isotopic compositions expected from such stars. This is a long-standing puzzle, which points to serious gaps in our understanding of the lifecycle of stars and dust in our Galaxy. Here we show that the increased proton-capture rate of $^{17}$O reported by a recent underground experiment leads to $^{17}$O/$^{16}$O isotopic ratios that match those observed in a population of stardust grains, for proton-burning temperatures of 60-80 million K. These temperatures are indeed achieved at the base of the convective envelope during the late evolution of intermediate-mass stars of 4-8 solar masses, which reveals them as the most likely site of origin of the grains. This result provides the first direct evidence that these stars contributed to the dust inventory from which the Solar System formed.