We forecast the reionization history constraints, inferred from Lyman-alpha damping wing absorption features, for a future sample of $sim 20$ $z geq 6$ gamma-ray burst (GRB) afterglows. We describe each afterglow spectrum by a three-parameter model. First, L characterizes the size of the ionized region (the bubble size) around a GRB host halo. Second, $langle{x_{rm HI}rangle}$ is the volume-averaged neutral fraction outside of the ionized bubble around the GRB, which is approximated as spatially uniform. Finally, $N_{mathrm{HI}}$ denotes the column-density of a local damped Lyman-alpha absorber (DLA) associated with the GRB host galaxy. The size distribution of ionized regions is extracted from a numerical simulation of reionization, and evolves strongly across the Epoch of Reionization (EoR). The model DLA column densities follow the empirical distribution determined from current GRB afterglow spectra. We use a Fisher matrix formalism to forecast the $langle{x_{rm HI}(z)rangle}$ constraints that can be obtained from follow-up spectroscopy of afterglows with SNR = 20 per R=3,000 resolution element at the continuum. We find that the neutral fraction may be determined to better than 10-15% (1-$sigma$) accuracy from this data across multiple independent redshift bins at $z sim 6-10$, spanning much of the EoR, although the precision degrades somewhat near the end of reionization. A more futuristic survey with $80$ GRB afterglows at $z geq 6$ can improve the precision here by a factor of $2$ and extend measurements out to $z sim 14$. We further discuss how these constraints may be combined with estimates of the escape fraction of ionizing photons, derived from the DLA column density distribution towards GRBs extracted at slightly lower redshift. This combination will help in testing whether we have an accurate census of the sources that reionized the universe.