For decades, cosmologists have been using galaxies to trace the large-scale distribution of matter. At present, the largest source of systematic uncertainty in this analysis is the challenge of modeling the complex relationship between galaxy redshift and the distribution of dark matter. If all galaxies sat in the centers of halos, there would be minimal Finger-of-God (FoG) effects and a simple relationship between the galaxy and matter distributions. However, many galaxies, even some of the luminous red galaxies (LRGs), do not lie in the centers of halos. Because the galaxy-galaxy lensing is also sensitive to the off-centered galaxies, we show that we can use the lensing measurements to determine the amplitude of this effect and to determine the expected amplitude of FoG effects. We develop an approach for using the lensing data to model how the FoG suppresses the power spectrum amplitudes and show that the current data implies a 30% suppression at wavenumber k=0.2h/Mpc. Our analysis implies that it is important to complement a spectroscopic survey with an imaging survey with sufficient depth and wide field coverage. Joint imaging and spectroscopic surveys allow a robust, unbiased use of the power spectrum amplitude information: it improves the marginalized error of growth rate fg=dln D/dln a by up to a factor of 2 over a wide range of redshifts z<1.4. We also find that the dark energy equation-of-state parameter, w0, and the neutrino mass, fnu, can be unbiasedly constrained by combining the lensing information, with an improvement of 10--25% compared to a spectroscopic survey without lensing calibration.