We report the discovery of a relic Giant Radio Galaxy (GRG) J021659-044920 at redshift $z sim 1.3$ that exhibits large-scale extended, nearly co-spatial, radio and X-ray emission from radio lobes, but no detection of Active Galactic Nuclei core, jets and hotspots. The total angular extent of the GRG at the observed frame 0.325 GHz, using Giant Metrewave Radio Telescope observations is found to be ${sim}$ 2.4 arcmin, that corresponds to a total projected linear size of $sim$ 1.2 Mpc. The integrated radio spectrum between 0.240 and 1.4 GHz shows high spectral curvature (${alpha}_{rm 0.610~GHz}^{rm 1.4~GHz} - {alpha}_{rm 0.240~GHz}^{rm 0.325~GHz}$ $>$ 1.19) with sharp steepening above 0.325 GHz, consistent with relic radio emission that is $sim$ 8 $times$ 10$^{6}$ yr old. The radio spectral index map between observed frame 0.325 and 1.4~GHz for the two lobes varies from 1.4 to 2.5 with the steepening trend from outer-end to inner-end, indicating backflow of plasma in the lobes. The extended X-ray emission characterized by an absorbed power-law with photon index $sim$ 1.86 favours inverse-Compton scattering of the Cosmic Microwave Background (ICCMB) photons as the plausible origin. Using both X-ray and radio fluxes under the assumption of ICCMB we estimate the magnetic field in the lobes to be 3.3 $mu$G. The magnetic field estimate based on energy equipartition is $sim$ 3.5 $mu$G. Our work presents a case study of a rare example of a GRG caught in dying phase in the distant Universe.