We present a detailed low temperature scanning tunneling microscopy study of the commensurate charge density wave (CDW) in 1$T$-TiSe$_2$ in the presence of single atom defects. We find no significant modification of the CDW lattice in single crystals with native defects concentrations where some bulk probes already measure substantial reductions in the CDW phase transition signature. Systematic analysis of STM micrographs combined with density functional theory modelling of atomic defect patterns indicate that the observed CDW modulation lies in the Se surface layer. The defect patterns clearly show there are no 2$H$-polytype inclusions in the CDW phase, as previously found at room temperature [Titov A.N. et al, Phys. Sol. State 53, 1073 (2011). They further provide an alternative explanation for the chiral Friedel oscillations recently reported in this compound [J. Ishioka et al., Phys. Rev. B 84, 245125, (2011)].