We model the evolution of the abundances of light elements in carbon-enhanced metal-poor (CEMP) stars, under the assumption that such stars are formed by mass transfer in a binary system. We have modelled the accretion of material ejected by an asymptotic giant branch star on to the surface of a companion star. We then examine three different scenarios: one in which the material is mixed only by convective processes, one in which thermohaline mixing is present and a third in which both thermohaline mixing and gravitational settling are taken in to account. The results of these runs are compared to light element abundance measurements in CEMP stars (primarily CEMP-s stars, which are rich in $s$-processes elements and likely to have formed by mass transfer from an AGB star), focusing on the elements Li, F, Na and Mg. None of the elements is able to provide a conclusive picture of the extent of mixing of accreted material. We confirm that lithium can only be preserved if little mixing takes place. The bulk of the sodium observations suggest that accreted material is effectively mixed but there are also several highly Na and Mg-rich objects that can only be explained if the accreted material is unmixed. We suggest that the available sodium data may hint that extra mixing is taking place on the giant branch, though we caution that the data is sparse.