We investigated the static and dynamic magnetic properties of the polar ferrimagnet Mn$_2$Mo$_3$O$_8$ in three magnetically ordered phases via magnetization, magnetic torque, and THz absorption spectroscopy measurements. The observed magnetic field dependence of the spin-wave resonances, including Brillouin zone-center and zone-boundary excitations, magnetization, and torque, are well described by an extended two-sublattice antiferromagnetic classical mean-field model. In this orbitally quenched system, the competing weak easy-plane and easy-axis single-ion anisotropies of the two crystallographic sites are determined from the model and assigned to the tetra- and octahedral sites, respectively, by ab initio calculations.