We probe local charge fluctuations in a semiconductor via laser spectroscopy on a nearby self-assembled quantum dot. We demonstrate that the quantum dot is sensitive to changes in the local environment at the single charge level. By controlling the charge state of localized defects, we are able to infer the distance of the defects from the quantum dot with +-5 nm resolution. The results identify and quantify the main source of charge noise in the commonly-used optical field-effect devices. Based on this understanding we achieve routinely close-totransform-limited quantum dot optical linewidths.