We present a survey for neutral atomic-carbon (CI) along gamma-ray burst (GRB) sightlines, which probes the shielded neutral gas-phase in the interstellar medium (ISM) of GRB host galaxies at high redshift. We compile a sample of 29 medium- to high-resolution GRB optical afterglow spectra spanning a redshift range through most of cosmic time from $1 < z < 6$. We find that seven ($approx 25%$) of the GRBs entering our statistical sample have CI detected in absorption. It is evident that there is a strong excess of cold gas in GRB hosts compared to absorbers in quasar sightlines. We investigate the dust properties of the GRB CI absorbers and find that the amount of neutral carbon is positively correlated with the visual extinction, $A_V$, and the strength of the 2175 AA dust extinction feature, $A_{mathrm{bump}}$. GRBs with CI detected in absorption are all observed above a certain threshold of $log N$(HI)$/mathrm{cm}^{-2}$ + [X/H] > 20.7 and a dust-phase iron column density of $log N$(Fe)$_{mathrm{dust}}/mathrm{cm}^{-2}$ > 16.2. In contrast to the SED-derived dust properties, the strength of the CI absorption does not correlate with the depletion-derived dust properties. This indicates that the GRB CI absorbers trace dusty systems where the dust composition is dominated by carbon-rich dust grains. The observed higher metal and dust column densities of the GRB CI absorbers compared to H$_2$- and CI-bearing quasar absorbers is mainly a consequence of how the two absorber populations are selected, but is also required in the presence of intense UV radiation fields in actively star-forming galaxies.