We have measured the impact of atomic hydrogen adsorption on the electronic transport properties of graphene sheets as a function of hydrogen coverage and initial, pre-hydrogenation field-effect mobility. Our results are compatible with hydrogen adsorbates inducing intervalley mixing by exerting a short-range scattering potential. The saturation coverages for different devices are found to be proportional to their initial mobility, indicating that the number of native scatterers is proportional to the saturation coverage of hydrogen. By extrapolating this proportionality, we show that the field-effect mobility can reach $1.5 times 10^4$ cm$^2$/V sec in the absence of the hydrogen-adsorbing sites. This affinity to hydrogen is the signature of the most dominant type of native scatterers in graphene-based field-effect transistors on SiO$_2$.