We present optical spectroscopy measurements on rare-earth ditelluride single crystals of LaTe$_{1.95}$ and CeTe$_{1.95-x}$Se$_x$ (x=0 and 0.16). The measurements reveal formation of charge density wave energy gaps at rather high energy levels, e.g. 2$Deltasim$ 8500 cm for LaTe$_{1.95}$, and 6800 cm for CeTe$_{1.95}$. More strikingly, the study reveals that, different from the rare-earth tri-tellurides, the Te vacancies and disorder effect play a key role in the low-energy charge excitations of ditelluride systems. Although an eminent peak is observed between 800 and 1500 cm in conductivity spectra for LaTe$_{1.95}$, and CeTe$_{1.95-x}$Se$_x$ (x=0. 0.16), our analysis indicates that it could not be attributed to the formation of a small energy gap, instead it could be well accounted for by the localization modified Drude model. Our study also indicates that the low-tempreature optical spectroscopic features are distinctly different from a semiconducting CDW state with entirely gapped Fermi surfaces.