Optical cavities provide high sensitivity to dispersion since their resonance frequencies depend on the index of refraction. We present a direct, broadband, and accurate measurement of the modes of a high finesse cavity using an optical frequency comb and a mechanical Fourier transform spectrometer with a kHz-level resolution. We characterize 16000 cavity modes spanning 16 THz of bandwidth in terms of center frequency, linewidth, and amplitude. We retrieve the group delay dispersion of the cavity mirror coatings and pure N${_2}$ with 0.1 fs${^2}$ precision and 1 fs${^2}$ accuracy, as well as the refractivity of the 3{ u}1+{ u}3 absorption band of CO${_2}$ with 5 x 10${^{-12}}$ precision. This opens up for broadband refractive index metrology and calibration-free spectroscopy of entire molecular bands.