Hexagonal boron nitride (hBN) is a wide bandgap van der Waals material that is emerging as a powerful platform for quantum optics and nanophotonics. In this work, we demonstrate whispering gallery mode silica microresonators hybridized with thin layers of epitaxially grown hBN that exhibit high optical quality factor $> 7 times 10^5$. Measurements of the effect of hBN thickness on optical $Q$ and comparison with a theoretical model allows the linear optical absorption coefficient of the hBN films to be estimated. These high-$Q$ devices will be useful for applications in quantum and nonlinear optics, and their hybridized geometry provides a sensitive platform for evaluating losses in hBN and other 2D materials.