We have stabilized an external cavity diode laser to a whispering gallery mode resonator formed by a protrusion of a single-crystal magnesiumdifluoride cylinder. The cylinders compact dimensions (<1 cm^3) reduce the sensitivity to vibrations and simplify the stabilization of its temperature in a compact setup. In a comparison to an ultrastable laser used for precision metrology we determine a minimum Allan deviation of 20 Hz, corresponding to a relative Allan deviation of 6*10^-14, at an integration time of 100 ms. This level of instability is compatible with the limits imposed by fundamental fluctuations of the materials refractive index at room temperature.