The effects on the local structure due to self-irradiation damage of Ga stabilized $delta$-Pu stored at cryogenic temperatures have been examined using extended x-ray absorption fine structure (EXAFS) experiments. Extensive damage, seen as a loss of local order, was evident after 72 days of storage below 15 K. The effect was observed from both the Pu and Ga sites, although less pronounced around Ga. Isochronal annealing was performed on this sample to study the annealing processes that occur between cryogenic and room temperature storage conditions, where damage is mostly reversed. Damage fractions at various points along the annealing curve have been determined using an amplitude-ratio method, standard EXAFS fitting, and a spherical crystallite model, and provide information complementary to previous electrical resistivity- and susceptibility-based isochronal annealing studies. The use of a spherical crystallite model accounts for the changes in EXAFS spectra using just two parameters, namely, the crystalline fraction and the particle radius. Together, these results are discussed in terms of changes to the local structure around Ga and Pu throughout the annealing process and highlight the unusual role of Ga in the behavior of the lowest temperature anneals.