We present a terahertz spectroscopic study of polar ferrimagnet FeZnMo$_3$O$_8$. Our main finding is a giant high-temperature optical diode effect, or nonreciprocal directional dichroism, where the transmitted light intensity in one direction is over 100 times lower than intensity transmitted in the opposite direction. The effect takes place in the paramagnetic phase with no long-range magnetic order in the crystal, which contrasts sharply with all existing reports of the terahertz optical diode effect in other magnetoelectric materials, where the long-range magnetic ordering is a necessary prerequisite. In fzmo, the effect occurs resonantly with a strong magnetic dipole active transition centered at 1.27 THz and assigned as electron spin resonance between the eigenstates of the single-ion anisotropy Hamiltonian. We propose that the optical diode effect in paramagnetic FeZnMo$_3$O$_8$ is driven by signle-ion terms in magnetoelectric free energy.