We study high-energy emission from the mergers of neutron star binaries as electromagnetic counterparts to gravitational waves aside from short gamma-ray bursts. The mergers entail significant mass ejection, which interacts with the surrounding medium to produce similar but brighter remnants than supernova remnants in a few years. We show that electrons accelerated in the remnants can produce synchrotron radiation in X-rays detectable at $sim 100$ Mpc by current generation telescopes and inverse Compton emission in gamma rays detectable by the emph{Fermi} Large Area Telescopes and the Cherenkov Telescope Array under favorable conditions. The remnants may have already appeared in high-energy surveys such as the Monitor of All-sky X-ray Image and the emph{Fermi} Large Area Telescope as unidentified sources. We also suggest that the merger remnants could be the origin of ultra-high-energy cosmic rays beyond the knee energy, $sim 10^{15}$ eV, in the cosmic-ray spectrum.