The oxygen dopants are essential in tuning electronic properties of Bi$_2$Sr$_2$Ca$_{n-1}$Cu$_n$O$_{2n+4+delta}$ superconductors. Here we apply the technique of scanning tunneling microscopy and spectroscopy to study the influence of oxygen dopants in an optimally doped Bi$_2$Sr$_{2-x}$La$_x$CuO$_{6+delta}$ and an overdoped Bi$_{2-y}$Pb$_y$Sr$_2$CuO$_{6+delta}$. In both samples, we find that interstitial oxygen atoms on the SrO layers dominate over the other two forms of oxygen dopants, oxygen vacancies on the SrO layers and interstitial oxygen atoms on the BiO layers. The hole doping is estimated from the oxygen concentration, as compared to the result extracted from the measured Fermi surface. The precise spatial location is employed to obtain a negative correlation between the oxygen dopants and the inhomogeneous pseudogap.