The lunar South pole likely contains significant amounts of water in the permanently shadowed craters there. Extracting this water for life support at a lunar base or to make rocket fuel would take large amounts of power, of order Gigawatts. A natural place to obtain this power are the Peaks of Eternal Light, that lie a few kilometers away on the crater rims and ridges above the permanently shadowed craters. The amount of solar power that could be captured depends on how tall a tower can be built to support the photovoltaic panels. The low gravity, lack of atmosphere, and quiet seismic environment of the Moon suggests that towers could be built much taller than on Earth. Here we look at the limits to building tall concrete towers on the Moon. We choose concrete as the capital cost of transporting large masses of iron or carbon fiber to the Moon is presently so expensive that profitable operation of a power plant is unlikely. Concrete instead can be manufactured in situ from the lunar regolith. We find that, with minimum wall thicknesses (20 cm), towers up to several kilometers tall are stable. The mass of concrete needed, however, grows rapidly with height, from $sim$ 760 mt at 1 km to $sim$ 4,100 mt at 2 km to $sim 10^5$ mt at 7 km and $sim 10^6$ mt at 17 km.