Observations with the Swift satellite of X-ray afterglows of more than a hundred gamma ray bursts (GRBs) with known redshift reveal ubiquitous soft X-ray absorption. The directly measured optical depth tau at a given observed energy is found to be constant on average at redshift z > 2, i.e., <tau (0.5 keV) >_{z > 2} = 0.40+/- 0.02. Such an asymptotic optical depth is expected if the foreground diffuse intergalactic medium (IGM) dominates the absorption effect, and if the metallicity of the diffuse IGM reaches ~ 0.2 - 0.4 solar at z = 0. To further test the IGM absorption hypothesis, we analyze the 12 highest S/N (> 5000 photon) z > 2 quasar spectra from the XMM-Newton archive, which are all extremely radio loud (RLQs). The quasar optical depths are found to be consistent with the mean GRB value. The four lowest-z quasars (2 < z < 2.5), however, do not show significant absorption. The best X-ray spectra of radio-quiet quasars (RQQs) at z > 2 provide only upper limits to the absorption, which are still consistent with the RLQs, albeit with much lower S/N (< 1000 photons at z ~ 4). Lack of quasar absorption poses a challenge to the smooth IGM interpretation, and could allude to the opacity being rather due to the jets in RLQs and GRBs. However, the jet absorbing column would need to appear in RLQs only at z > 2.5, and in GRBs to strongly increase with z in order to produce the observed tendency to a constant mean tau. High X-ray spectral resolution can differentiate between an absorber intrinsic to the source that produces discernible spectral lines, and the diffuse IGM that produces significant absorption, but no discrete features.