We present the largest homogeneous survey of $z>4.4$ damped Lyman-$alpha$ systems (DLAs) using the spectra of 163 QSOs that comprise the Giant Gemini GMOS (GGG) survey. With this survey we make the most precise high-redshift measurement of the cosmological mass density of neutral hydrogen, $Omega_{rm HI}$. At such high redshift important systematic uncertainties in the identification of DLAs are produced by strong intergalactic medium absorption and QSO continuum placement. These can cause spurious DLA detections, result in real DLAs being missed, or bias the inferred DLA column density distribution. We correct for these effects using a combination of mock and higher-resolution spectra, and show that for the GGG DLA sample the uncertainties introduced are smaller than the statistical errors on $Omega_{rm HI}$. We find $Omega_{rm HI}=0.98^{+0.20}_{-0.18}times10^{-3}$ at $langle zrangle=4.9$, assuming a 20% contribution from lower column density systems below the DLA threshold. By comparing to literature measurements at lower redshifts, we show that $Omega_{rm HI}$ can be described by the functional form $Omega_{rm HI}(z)propto(1+z)^{0.4}$. This gradual decrease from $z=5$ to $0$ is consistent with the bulk of HI gas being a transitory phase fuelling star formation, which is continually replenished by more highly-ionized gas from the intergalactic medium, and from recycled galactic winds.