We study the chemical abundances of the interstellar medium surrounding high z gamma-ray bursts (GRBs) through analysis of the damped Lya systems (DLAs) identified in afterglow spectra. These GRB-DLAs are characterized by large HI column densities N(HI) and metallicities [M/H] spanning 1/100 to nearly solar, with median [M/H]>-1. The majority of GRB-DLAs have [M/H] values exceeding the cosmic mean metallicity of atomic gas at z>2, i.e. if anything, the GRB-DLAs are biased to larger metallicity. We also observe (i) large [Zn/Fe] values (>+0.6) and sub-solar Ti/Fe ratios which imply substantial differential depletion, (ii) large a/Fe ratios suggesting nucleosynthetic enrichment by massive stars, and (iii) low C^0/C^+ ratios (<10^{-4}). Quantitatively, the observed depletion levels and C^0/C^+ ratios of the gas are not characteristic of cold, dense HI clouds in the Galactic ISM. We argue that the GRB-DLAs represent the ISM near the GRB but not gas directly local to the GRB (e.g. its molecular cloud or circumstellar material). We compare these observations with DLAs intervening background quasars (QSO-DLAs). The GRB-DLAs exhibit larger N(HI) values, higher a/Fe and Zn/Fe ratios, and have higher metallicity than the QSO-DLAs. We argue that the differences primarily result from galactocentric radius-dependent differences in the ISM: GRB-DLAs preferentially probe denser, more depleted, higher metallicity gaslocated in the inner few kpc whereas QSO-DLAs are more likely to intersect the less dense, less enriched, outer regions of the galaxy. Finally, we investigate whether dust obscuration may exclude GRB-DLA sightlines from QSO-DLA samples; we find that the majority of GRB-DLAs would be recovered which implies little observational bias against large N(HI) systems.