We present a polarization-resolved, high-resolution Raman scattering study of the three consecutive charge density wave (CDW) regimes in $1T$-TaS$_2$ single crystals, supported by textit{ab initio} calculations. Our analysis of the spectra within the low-temperature commensurate (C-CDW) regime shows $mathrm{P3}$ symmetry of the system, thus excluding the previously proposed triclinic stacking of the star-of-David structure, and promoting trigonal or hexagonal stacking instead. The spectra of the high-temperature incommensurate (IC-CDW) phase directly project the phonon density of states due to the breaking of the translational invariance, supplemented by sizeable electron-phonon coupling. Between 200 and 352,K, our Raman spectra show contributions from both the IC-CDW and the C-CDW phase, indicating their coexistence in the so-called nearly-commensurate (NC-CDW) phase. The temperature-dependence of the symmetry-resolved Raman conductivity indicates the stepwise reduction of the density of states in the CDW phases, followed by a Mott transition within the C-CDW phase. We determine the size of the Mott gap to be $Omega_{rm gap}approx 170-190$ meV, and track its temperature dependence.