Many clays, soils, biological tissues, foods, and coatings are shrinkable, granular materials: they are composed of packed, hydrated grains that shrink when dried. In many cases, these packings crack during drying, critically hindering applications. However, while cracking has been widely studied for bulk gels and packings of non-shrinkable grains, little is known about how packings of shrinkable grains crack. Here, we elucidate how grain shrinkage alters cracking during drying. Using experiments with model shrinkable hydrogel beads, we show that differential shrinkage can dramatically alter crack evolution during drying---in some cases, even causing cracks to spontaneously self-close. In other cases, packings shrink without cracking or crack irreversibly. We developed both granular and continuum models to quantify the interplay between grain shrinkage, poromechanics, packing size, drying rate, capillarity, and substrate friction on cracking. Guided by the theory, we also found that cracking can be completely altered by varying the spatial profile of drying. Our work elucidates the rich physics underlying cracking in shrinkable, granular packings, and yields new strategies for controlling crack evolution.