Graphite has been used as neutron moderator or reflector in many nuclear reactors. The irradiation of graphite in a nuclear reactor results in a complex population of defects. Heating of the irradiated graphite at high temperatures results in annihilation of the defects with release of an unusually large energy, called the Wigner energy. From various experiments on highly irradiated graphite samples from CIRUS reactor at Trombay and ab-initio simulations, we have for the first time identified various 2-, 3- and 4-coordinated topological structures in defected graphite, and provided microscopic mechanism of defect annihilation on heating and release of the Wigner energy. The annihilation process involves cascading cooperative movement of atoms in two steps involving an intermediate structure. Our work provides new insights in understanding of the defect topologies and annihilation in graphite which is of considerable importance to wider areas of graphitic materials including graphene and carbon nanotubes.