PtSe2 is attracting considerable attention as a high mobility two-dimensional material with envisionned applications in microelectronics, photodetection and spintronics. The growth of high quality PtSe2 on insulating substrates with wafer-scale uniformity is a prerequisite for electronic transport investigations and practical use in devices. Here, we report the growth of highly oriented few-layers PtSe2 on ZnO(0001) by molecular beam epitaxy. The crystalline structure of the films is characterized with electron and X-ray diffraction, atomic force microscopy and transmission electron microscopy. The comparison with PtSe2 layers grown on graphene, sapphire, mica, SiO2 and Pt(111) shows that among insulating substrates, ZnO(0001) yields films of superior structural quality. Hall measurements performed on epitaxial ZnO/PtSe2 with 5 monolayers of PtSe2 show a clear semiconducting behaviour and a high mobility in excess of 200 cm2V 1s-1 at room temperature and up to 447 cm2V-1s-1 at low temperature.