Vertically stacked van der Waals heterostructures are a lucrative platform for exploring the rich electronic and optoelectronic phenomena in two-dimensional materials. Their performance will be strongly affected by impurities and defects at the interfaces. Here we present the first systematic study of interfaces in van der Waals heterostructure using cross sectional scanning transmission electron microscope (STEM) imaging. By measuring interlayer separations and comparing these to density functional theory (DFT) calculations we find that pristine interfaces exist between hBN and MoS2 or WS2 for stacks prepared by mechanical exfoliation in air. However, for two technologically important transition metal dichalcogenide (TMDC) systems, MoSe2 and WSe2, our measurement of interlayer separations provide the first evidence for impurity species being trapped at buried interfaces with hBN: interfaces which are flat at the nanometer length scale. While decreasing the thickness of encapsulated WSe2 from bulk to monolayer we see a systematic increase in the interlayer separation. We attribute these differences to the thinnest TMDC flakes being flexible and hence able to deform mechanically around a sparse population of protruding interfacial impurities. We show that the air sensitive two dimensional (2D) crystal NbSe2 can be fabricated into heterostructures with pristine interfaces by processing in an inert-gas environment. Finally we find that adopting glove-box transfer significantly improves the quality of interfaces for WSe2 compared to processing in air.