We study a coupled dark energy scenario in which a massive vector field $A_{mu}$ with broken $U(1)$ gauge symmetry interacts with the four-velocity $u_c^{mu}$ of cold dark matter (CDM) through the scalar product $Z=-u_c^{mu} A_{mu}$. This new coupling corresponds to the momentum transfer, so that the background vector and CDM continuity equations do not have explicit interacting terms analogous to the energy exchange. Hence the observational preference of uncoupled generalized Proca theories over the $Lambda$CDM model can be still maintained at the background level. Meanwhile, the same coupling strongly affects the evolution of cosmological perturbations. While the effective sound speed of CDM vanishes, the propagation speed and no-ghost condition of a longitudinal scalar of $A_{mu}$ and the CDM no-ghost condition are subject to nontrivial modifications by the $Z$ dependence in the Lagrangian. We propose a concrete dark energy model and show that the gravitational interaction on scales relevant to the linear growth of large-scale structures can be smaller than the Newton constant at low redshifts. This leads to the suppression of growth rates of both CDM and total matter density perturbations, so our model allows an interesting possibility for reducing the tension of matter density contrast $sigma_8$ between high- and low-redshift measurements.