We investigate a neutral gauge boson X originated from a hidden U(1) extension of the standard model as the particle dark matter candidate. The vector dark matter interacts with the standard model fermions through heavy fermion mediators. The interactions give rise to t-channel annihilation cross section in the XX to ff process, which dominates the thermal relic abundance during thermal freeze-out and produces measurable gamma-ray flux in the galactic halo. For a light vector dark matter, if it predominantly couples to the third generation fermions, this model could explain the excess of gamma rays from the galactic center. We show that the vector dark matter with a mass of 20 ~ 40 GeV and that annihilate into the bb and tautau final states provides an excellent description of the observed gamma-ray excess. The parameter space aimed at explaining the gamma-ray excess, could also provide the correct thermal relic density and is compatible with the constraints from electroweak precision data, Higgs invisible decay, and collider searches. We also show the dark matter couplings to the nucleon from the fermion portal interactions are loop-suppressed, and only contribute to the spin-dependent cross section. So the vector dark matter could easily escape the stringent constraints from the direct detection experiments.