We report spin-split Landau levels of quasi-two-dimensional Dirac fermions in a layered antiferromagnet EuMnBi$_2$, as revealed by interlayer resistivity measurements in a tilted magnetic field up to $sim$35 T. The amplitude of Shubnikov-de Haas (SdH) oscillation in interlayer resistivity is strongly modulated by changing the tilt angle of the field, i.e., the Zeeman-to-cyclotron energy ratio. The effective $g$ factor estimated from the tilt angle, where the SdH oscillation exhibits a phase inversion, differs by approximately 50% between two antiferromagnetic phases. This observation signifies a marked impact of the magnetic order of Eu sublattice on the Dirac-like band structure. The origin may be sought in strong exchange coupling with the local Eu moments, as verified by the first-principles calculation.