Voyager 2 observations revealed that the internal luminosity of Neptune is an order of magnitude higher than that of Uranus. If the two planets have similar interior structures and cooling histories, the luminosity of Neptune can only be explained by invoking some energy source beyond gravitational contraction. This paper investigates whether Centaur impacts could provide the energy necessary to produce the luminosity of Neptune. The major findings are (1) that impacts on both Uranus and Neptune are too infrequent to provide luminosities of order the observed value for Neptune, even for optimistic impact-rate estimates, and (2) that Uranus and Neptune rarely have significantly different impact-generated luminosities at any given time. Uranus and Neptune most likely have structural differences that force them to cool and contract at different rates.