We discuss the high enegry afterglow emission (including high energy photons, neutrinos and cosmic rays) following the 2004 December 27 Giant Flare from SGR 1806-20. If the initial outflow is relativistic with a bulk Lorentz factor Gamma_0sim {rm tens}, the high-energy tail of the synchrotron emission from electrons in the forward shock region gives rise to a prominent sub-GeV emission, if the electron spectrum is hard enough and if the intial Lorentz factor is high enough. This signal could serve as a diagnosis of the initial Lorentz factor of the giant flare outflow. This component is potentially detectable by GLAST if a similar giant flare occurs in the GLAST era. With the available 10 MeV data, we constrain that Gamma_0 < 50 if the electron distribution is a single power law. For a broken power law distribution of electrons, a higher Gamma_0 is allowed. At energies higher than 1 GeV, the flux is lower because of a high energy cut off of the synchrotron emission component. The synchrotron self-Compton emission component and the inverse Compton scattering component off the photons in the giant flare oscillation tail are also considered, but they are found not significant given a moderate Gamma_0 (e.g. leq 10). The forward shock also accelerates cosmic rays to the maximum energy 10^{17}eV, and generate neutrinos with a typical energy 10^{14}eV through photomeson interaction with the X-ray tail photons. However, they are too weak to be detectable.