We present a soft x-ray angle-resolved photoemission spectroscopy study of the overdoped high-temperature superconductors La$_{2-x}$Sr$_x$CuO$_4$ and La$_{1.8-x}$Eu$_{0.2}$Sr$_x$CuO$_4$. In-plane and out-of-plane components of the Fermi surface are mapped by varying the photoemission angle and the incident photon energy. No $k_z$ dispersion is observed along the nodal direction, whereas a significant antinodal $k_z$ dispersion is identified. Based on a tight-binding parametrization, we discuss the implications for the density of states near the van-Hove singularity. Our results suggest that the large electronic specific heat found in overdoped La$_{2-x}$Sr$_x$CuO$_4$ can not be assigned to the van-Hove singularity alone. We therefore propose quantum criticality induced by a collapsing pseudogap phase as a plausible explanation for observed enhancement of electronic specific heat.