The parkerite-type ternary chalcogenide Bi$_2$Rh$_3$Se$_2$ was discovered to be a charge density wave (CDW) superconductor. However, there was a debate on whether the observed phase transition at 240 K could be attributed to the formation of CDW order. To address the issue, we performed optical spectroscopy and ultrafast pump-probe measurements on single crystal samples of Bi$_2$Rh$_3$Se$_2$. Our optical conductivity measurement reveals clearly the formation of an energy gap with associated spectral change only at low energies, yielding strong evidence for a CDW phase transition at 240 K. Time resolved pump-probe measurement provides further support for the CDW phase transition. The amplitude and relaxation time of quasiparticles extracted from the photoinduced reflectivity show strong enhancement near transition temperature, yielding further evidence for the CDW energy gap formation. Additionally, a collective mode is identified from the oscillations in the pump-probe time delay at low temperature. This mode, whose frequency decreases gradually at elevated temperature, could be naturally attributed to the amplitude mode of CDW state.