Using data from the COSMOS survey, we perform the first joint analysis of galaxy-galaxy weak lensing, galaxy spatial clustering, and galaxy number densities. Carefully accounting for sample variance and for scatter between stellar and halo mass, we model all three observables simultaneously using a novel and self-consistent theoretical framework. Our results provide strong constraints on the shape and redshift evolution of the stellar-to-halo mass relation (SHMR) from z=0.2 to z=1. At low stellar mass, we find that halo mass scales as Mh M*^0.46 and that this scaling does not evolve significantly with redshift to z=1. We show that the dark-to-stellar ratio, Mh/M*, varies from low to high masses, reaching a minimum of Mh/M*~27 at M*=4.5x10^10 Msun and Mh=1.2x10^12 Msun. This minimum is important for models of galaxy formation because it marks the mass at which the accumulated stellar growth of the central galaxy has been the most efficient. We describe the SHMR at this minimum in terms of the pivot stellar mass, M*piv, the pivot halo mass, Mhpiv, and the pivot ratio, (Mh/M*)piv. Thanks to a homogeneous analysis of a single data set, we report the first detection of mass downsizing trends for both Mhpiv and M*piv. The pivot stellar mass decreases from M*piv=5.75+-0.13x10^10 Msun at z=0.88 to M*piv=3.55+-0.17x10^10 Msun at z=0.37. Intriguingly, however, the corresponding evolution of Mhpiv leaves the pivot ratio constant with redshift at (Mh/M*)piv~27. We use simple arguments to show how this result raises the possibility that star formation quenching may ultimately depend on Mh/M* and not simply Mh, as is commonly assumed. We show that simple models with such a dependence naturally lead to downsizing in the sites of star formation. Finally, we discuss the implications of our results in the context of popular quenching models, including disk instabilities and AGN feedback.