We use a combined generalized spectroscopic ellipsometry and density functional theory approach to determine and analyze the anisotropic dielectric functions of an $alpha$-Ga$_2$O$_3$ thin film. The sample is grown epitaxially by plasma-assisted molecular beam epitaxy on $m$-plane sapphire. Generalized spectroscopic ellipsometry data from multiple sample azimuths in the spectral range from 0.73 eV to 8.75 eV are simultaneously analyzed. Density functional theory is used to calculate the valence and conduction band structure. We identify, for the indirect-bandgap material, two direct band-to-band transitions with $M_0$-type van Hove singularities for polarization perpendicular to the $c$ axis, $E_{0,perp}=5.46(6)$ eV and $E_{0,perp}=6.04(1)$ eV, and one direct band-to-band transition with $M_1$-type van Hove singularity for polarization parallel with $E_{0,||}=5.44(2)$ eV. We further identify excitonic contributions with small binding energy of 7 meV associated with the lowest ordinary transition, and a hyperbolic exciton at the $M_1$-type critical point with large binding energy of 178 meV.