Next-generation missions designed to detect biosignatures on exoplanets will also be capable of placing constraints on the presence of technosignatures (evidence for technological life) on these same worlds. Here, I estimate the detectability of nightside city lights on habitable, Earth-like, exoplanets around nearby stars using direct-imaging observations from the proposed LUVOIR and HabEx observatories. I use data from the Soumi National Polar-orbiting Partnership satellite to determine the surface flux from city lights at the top of Earths atmosphere, and the spectra of commercially available high-power lamps to model the spectral energy distribution of the city lights. I consider how the detectability scales with urbanization fraction: from Earths value of 0.05%, up to the limiting case of an ecumenopolis -- or planet-wide city. I then calculate the minimum detectable urbanization fraction using 300 hours of observing time for generic Earth-analogs around stars within 8 pc of the Sun, and for nearby known potentially habitable planets. Though Earth itself would not be detectable by LUVOIR or HabEx, planets around M-dwarfs close to the Sun would show detectable signals from city lights for urbanization levels of 0.4% to 3%, while city lights on planets around nearby Sun-like stars would be detectable at urbanization levels of $gtrsim10%$. The known planet Proxima b is a particularly compelling target for LUVOIR A observations, which would be able to detect city lights twelve times that of Earth in 300 hours, an urbanization level that is expected to occur on Earth around the mid-22nd-century. An ecumenopolis, or planet-wide city, would be detectable around roughly 50 nearby stars by both LUVOIR and HabEx, and a survey of these systems would place a $1,sigma$ upper limit of $lesssim2%$ on the frequency of ecumenopolis planets in the Solar neighborhood assuming no detections.