Yttrium orthosilicate (Y$_2$SiO$_5$, or YSO) has proved to be a convenient host for rare-earth ions used in demonstrations of microwave quantum memories and optical memories with microwave interfaces, and shows promise for coherent microwave--optical conversion owing to its favourable optical and spin properties. The strong coupling required by such microwave applications could be achieved using superconducting resonators patterned directly on Y$_2$SiO$_5$, and hence we investigate here the use of Y$_2$SiO$_5$ as an alternative to sapphire or silicon substrates for superconducting hybrid device fabrication. A NbN resonator with frequency 6.008 GHz and low power quality factor $Q approx 400000$ was fabricated on a Y$_2$SiO$_5$ substrate doped with isotopically enriched Nd$^{145}$. Measurements of dielectric loss yield a loss-tangent $tandelta = 4 times 10^{-6}$, comparable to sapphire. Electron spin resonance (ESR) measurements performed using the resonator show the characteristic angular dependence expected from the anisotropic Nd$^{145}$ spin, and the coupling strength between resonator and electron spins is in the high cooperativity regime ($C = 30$). These results demonstrate Y$_2$SiO$_5$ as an excellent substrate for low-loss, high-Q microwave resonators, especially in applications for coupling to optically-accessible rare earth spins.