We present and analyze four frequency measurements designed to characterize the performance of an optical frequency reference based on spectral hole burning in EuYSO. The first frequency comparison, between a single unperturbed spectral hole and a hy
drogen maser, demonstrates a fractional frequency drift rate of $5 times 10^{-18}$ s$^{-1}$. Optical-frequency comparisons between a pattern of spectral holes, a Fabry-Perot cavity, and an Al$^+$ optical atomic clock show a short-term fractional frequency stability of $1 times10^{-15} tau^{-1/2}$ that averages down to $2.5^{+1.1}_{-0.5} times 10^{-16}$ at $tau = 540~s$ (with linear frequency drift removed). Finally, spectral hole patterns in two different EuYSO crystals located in the same cryogenic vessel are compared, yielding a short-term stability of $7 times10^{-16} tau^{-1/2}$ that averages down to $5.5^{+1.8}_{-0.9} times 10^{-17}$ at $tau = 204$~s (with quadratic frequency drift removed).
