The impact of magnetic anisotropy on the skyrmion lattice (SkL) state in cubic chiral magnets has been overlooked for long, partly because a semi-quantitative description of the thermodynamically stable SkL phase pocket forming near the Curie temperature could be achieved without invoking anisotropy effects. However, there has been a range of phenomena reported recently in these materials, such as the formation of low-temperature tilted conical and SkL states as well as temperature-induced transformations of lattice geometry in metastable SkL states, where anisotropy was suspected to play a key role. To settle this issue on experimental basis, we quantified the cubic anisotropy in a series of CoZnMn-type cubic chiral magnets. We found that the strength of anisotropy is highly enhanced towards low temperatures in all the compounds, moreover, not only the magnitude but also the character of cubic anisotropy drastically varies upon changing the Co/Mn ratio. We correlate these changes with temperature- and composition-induced variations of the helical modulation vectors, the anharmonicity and structural rearrangements of the metastable SkLs and the spin relaxation rates. Similar systematic studies on magnetic anisotropy may not only pave the way for a quantitative and unified description of the stable and metastable modulated spin textures in cubic chiral magnets but would also help exploring further topological spin textures in this large class of skyrmion hosts.