Prediction of the soft X-ray absorption along lines of sight through our Galaxy is crucial for understanding the spectra of extragalactic sources, but requires a good estimate of the foreground column density of photoelectric absorbing species. Assuming uniform elemental abundances this reduces to having a good estimate of the total hydrogen column density, N(Htot)=N(HI)+2N(H2). The atomic component, N(HI), is reliably provided using the mapped 21 cm radio emission but estimating the molecular hydrogen column density, N(H2), expected for any particular direction, is difficult. The X-ray afterglows of GRBs are ideal sources to probe X-ray absorption in our Galaxy because they are extragalactic, numerous, bright, have simple spectra and occur randomly across the entire sky. We describe an empirical method, utilizing 493 afterglows detected by the Swift XRT, to determine N(Htot) through the Milky Way which provides an improved estimate of the X-ray absorption in our Galaxy and thereby leads to more reliable measurements of the intrinsic X-ray absorption and, potentially, other spectral parameters, for extragalactic X-ray sources. We derive a simple function, dependent on the product of the atomic hydrogen column density, N(HI), and dust extinction, E(B-V), which describes the variation of the molecular hydrogen column density, N(H2), of our Galaxy, over the sky. Using the resulting N(Htot) we show that the dust-to-hydrogen ratio is correlated with the carbon monoxide emission and use this ratio to estimate the fraction of material which forms interstellar dust grains. Our resulting recipe represents a significant revision in Galactic absorption compared to previous standard methods, particularly at low Galactic latitudes.