Lyman Limit systems (LLSs) trace the low-density circumgalactic medium and the most dense regions of the intergalactic medium, so their number density and evolution at high redshift, just after reionisation, are important to constrain. We present a survey for LLSs at high redshifts, $z_{rm LLS} =3.5$--5.4, in the homogeneous dataset of 153 optical quasar spectra at $z sim 5$ from the Giant Gemini GMOS survey. Our analysis includes detailed investigation of survey biases using mock spectra which provide important corrections to the raw measurements. We estimate the incidence of LLSs per unit redshift at $z approx 4.4$ to be $ell(z) = 2.6 pm 0.4$. Combining our results with previous surveys at $z_{rm LLS} <4$, the best-fit power-law evolution is $ell(z) = ell_* [(1+z)/4]^alpha$ with $ell_* = 1.46 pm 0.11$ and $alpha = 1.70 pm 0.22$ (68% confidence intervals). Despite hints in previous $z_{rm LLS} <4$ results, there is no indication for a deviation from this single power-law soon after reionization. Finally, we integrate our new results with previous surveys of the intergalactic and circumgalactic media to constrain the hydrogen column density distribution function, $f(N_{rm HI},X)$, over 10 orders of magnitude. The data at $z sim 5$ are not well described by the $f(N_{rm HI},X)$ model previously reported for $z sim 2$--3 (after re-scaling) and a 7-pivot model fitting the full $z sim 2$--5 dataset is statistically unacceptable. We conclude that there is significant evolution in the shape of $f(N_{rm HI},X)$ over this $sim$2 billion year period.