We use a large sample of galaxies at z~3 to establish a relationship between reddening, neutral gas covering fraction (fcov(HI)), and the escape of ionizing photons at high redshift. Our sample includes 933 galaxies at z~3, 121 of which have very deep spectroscopic observations (>7 hrs) in the rest-UV (lambda=850-1300 A) with Keck/LRIS. Based on the high covering fraction of outflowing optically-thick HI indicated by the composite spectra of these galaxies, we conclude that photoelectric absorption, rather than dust attenuation, dominates the depletion of ionizing photons. By modeling the composite spectra as the combination of an unattenuated stellar spectrum including nebular continuum emission with one that is absorbed by HI and reddened by a line-of-sight extinction, we derive an empirical relationship between E(B-V) and fcov(HI). Galaxies with redder UV continua have larger covering fractions of HI characterized by higher line-of-sight extinctions. Our results are consistent with the escape of Lya through gas-free lines-of-sight. Covering fractions based on low-ionization interstellar absorption lines systematically underpredict those deduced from the HI lines, suggesting that much of the outflowing gas may be metal-poor. We develop a model which connects the ionizing escape fraction with E(B-V), and which may be used to estimate the escape fraction for an ensemble of high-redshift galaxies. Alternatively, direct measurements of the escape fraction for our data allow us to constrain the intrinsic 900-to-1500 A flux density ratio to be >0.20, a value that favors stellar population models that include weaker stellar winds, a flatter initial mass function, and/or binary evolution. Lastly, we demonstrate how the framework discussed here may be used to assess the pathways by which ionizing radiation escapes from high-redshift galaxies. [Abridged]