We investigate Landau-quantized excitonic absorption and luminescence of monolayer WSe$_2$ under magnetic field. We observe gate-dependent quantum oscillations in the bright exciton and trions (or exciton-polarons) as well as the dark trions and their phonon replicas. Our results reveal spin- and valley-polarized Landau levels (LLs) with filling factors $n = +0, +1$ in the bottom conduction band and $n = -0$ to $-6$ in the top valence band, including the Berry-curvature-induced $n = pm0$ LLs of massive Dirac fermions. The LL filling produces periodic plateaus in the exciton energy shift accompanied by sharp oscillations in the exciton absorption width and magnitude. This peculiar exciton behavior can be simulated by semi-empirical calculations. The experimentally deduced g-factors of the conduction band (g ~ 2.5) and valence band (g ~ 15) exceed those predicted in a single-particle model (g = 1.5, 5.5, respectively). Such g-factor enhancement implies strong many-body interactions in gated monolayer WSe$_2$. The complex interplay between Landau quantization, excitonic effects, and many-body interactions makes monolayer WSe$_2$ a promising platform to explore novel correlated quantum phenomena.