Optical whispering gallery mode (WGM) resonators have been very attracting platforms for versatile Kerr frequency comb generations. We report a systematic study on the material dispersion of various optical materials that are capable of supporting quality factors above $10^9$. Using an analytical approximation of WGM resonant frequencies in disk resonators, we investigate the effect of the geometry and transverse mode order on the total group-velocity dispersion ($GVD$). We demonstrate that the major radii and the radial mode indices play an important role in tailoring the $GVD$ of WGM resonators. In particular, our study shows that in WGM disk-resonators, the polar families of modes have very similar $GVD$, while the radial families of modes feature dispersion values that can differ by up to several orders of magnitude. The effect of these giant dispersion shifts are experimentally evidenced in Kerr comb generation with magnesium fluoride. From a more general perspective, this critical feature enables to push the zero-dispersion wavelength of fluorite crystals towards the mid-infrared (mid-IR) range, thereby allowing for efficient Kerr comb generation in that spectral range. We show that barium fluoride is the most interesting crystal in this regard, due to its zero dispersion wavelength ($ZDW$) at $1.93 rm{mu m}$ and an optimal dispersion profile in the mid-IR regime. We expect our results to facilitate the design of different platforms for Kerr frequency comb generations in both telecommunication and mid-IR spectral ranges.