Objects falling into an overdensity appear larger on its near side and smaller on its far side than other objects at the same redshift. This produces a dipolar pattern of magnification, primarily as a consequence of the Doppler effect. At low redshift this Doppler magnification completely dominates the usual integrated gravitational lensing contribution to the lensing magnification. We show that one can optimally observe this pattern by extracting the dipole in the cross-correlation of number counts and galaxy sizes. This dipole allows us to almost completely remove the contribution from gravitational lensing up to redshift 0.5, and even at high redshift z~1 the dipole picks up the Doppler magnification predominantly. Doppler magnification should be easily detectable in current and upcoming optical and radio surveys; by forecasting for telescopes such as the SKA, we show that this technique is competitive with using peculiar velocities via redshift-space distortions to constrain dark energy. It produces similar yet complementary constraints on the cosmological model to those found using measurements of the cosmic shear.