Layered materials have uncommonly anisotropic thermal properties due to their strong in-plane covalent bonds and weak out-of-plane van der Waals interactions. Here we examine heat flow in graphene (graphite), h-BN, MoS2, and WS2 monolayers and bulk films, from diffusive to ballistic limits. We determine the ballistic thermal conductance limit (Gball) both in-plane and out-of-plane, based on full phonon dispersions from first-principles calculations. An overall phonon mean free path ({lambda}) is expressed in terms of Gball and the diffusive thermal conductivity, consistent with kinetic theory if proper averaging of phonon group velocity is used. We obtain a size-dependent thermal conductivity k(L) in agreement with available experiments, and find that k(L) only converges to >90% of the diffusive thermal conductivity for sample sizes L > 16{lambda}, which ranges from ~140 nm for MoS2 cross-plane to ~10 um for suspended graphene in-plane. These results provide a deeper understanding of microscopic thermal transport, revealing that device scales below which thermal size effects should be taken into account are generally larger than previously thought.