The nature of the first unidentified VHE gamma-ray source with significant angular offset from the Galactic plane of 3.5 degrees, HESS J1507-622, is explored. Fermi-LAT data in the high-energy (HE, 100 MeV < E < 100 GeV) gamma-ray range collected over 34 month are used to describe the spectral energy distribution (SED) of the source. HESS J1507-622 is detected in the Fermi energy range and its spectrum is best described by a power law in energy with Gamma=1.7 +/- 0.1 stat +/- 0.2_sys and integral flux between (0.3-300) GeV of F = (2.0 +/-0.5_stat +/- 1.0_sys) x 10^-9 cm^-2 s^-1. With the available data it is not possible to discriminate between a hadronic and a leptonic scenario for HESS J1507-622. The location and compactness of the source indicate a considerable physical offset from the Galactic plane for this object. In case of a multiple-kpc distance, this challenges a pulsar wind nebula (PWN) origin for HESS J1507-622 since the time of travel for a pulsar born in the Galactic disk to reach such a location would exceed the inverse Compton (IC) cooling time of electrons that are energetic enough to produce VHE gamma-rays. However, an origin of this gamma-ray source connected to a pulsar that was born off the Galactic plane in the explosion of a hypervelocity star cannot be excluded. The nature of HESS J1507-622 is still unknown to date, and a PWN scenario cannot be ruled out in general. On the contrary HESS J1507-622 could be the first discovered representative of a population of spatially extended VHE gamma-ray emitters with HE gamma-ray counterpart that are located at considerable offsets from the Galactic plane. Future surveys in the VHE gamma-ray range are necessary to probe the presence or absence of such a source population.(abridged)