Scalable fabrication of magnetic 2D materials and heterostructures constitutes a crucial step for scaling down current spintronic devices and the development of novel spintronic applications. Here, we report on van der Waals (vdW) epitaxy of the layered magnetic metal Fe$_3$GeTe$_2$ - a 2D crystal with highly tunable properties and a high prospect for room temperature ferromagnetism - directly on graphene by employing molecular beam epitaxy. Morphological and structural characterization confirmed the realization of large-area, continuous Fe$_3$GeTe$_2$/graphene heterostructure films with stable interfaces and good crystalline quality. Furthermore, magneto-transport and X-ray magnetic circular dichroism investigations confirmed a robust out-of-plane ferromagnetism in the layers, comparable to state-of-the-art exfoliated flakes from bulk crystals. These results are highly relevant for further research on wafer-scale growth of vdW heterostructures combining Fe$_3$GeTe$_2$ with other layered crystals such as transition metal dichalcogenides for the realization of multifunctional, atomically thin devices.