We combine femtosecond electron diffuse scattering experiments and first-principles calculations of the coupled electron-phonon dynamics to provide a detailed momentum-resolved picture of the ultrafast lattice thermalization in a thin film of black phosphorus. The measurements reveal the emergence of highly anisotropic non-thermal phonon populations which persist for several picoseconds following excitation of the electrons with a light pulse. Combining ultrafast dynamics simulations based on the time-dependent Boltzmann formalism and calculations of the structure factor, we reproduce the experimental data and identify the vibrational modes primarily responsible for the carrier relaxation via electron-phonon coupling and the subsequent lattice thermalization via phonon-phonon scattering. In particular, we attribute the non-equilibrium lattice dynamics of black phosphorus to highly-anisotropic phonon-assisted scattering processes, which are primarily mediated by high-energy optical phonons. Our approach paves the way towards unravelling and controlling microscopic energy-flow pathways in two-dimensional materials and van der Waals heterostructures, and may also be extended to other non-equilibrium phenomena involving coupled electron-phonon dynamics such as superconductivity, phase transitions or polaron physics.