The Sun is the only star whose surface can be directly resolved at high resolution, and therefore constitutes an excellent test case to explore the physical origin of stellar radial-velocity (RV) variability. We present HARPS observations of sunlight scattered off the bright asteroid 4/Vesta, from which we deduced the Suns activity-driven RV variations. In parallel, the HMI instrument onboard the Solar Dynamics Observatory provided us with simultaneous high spatial resolution magnetograms, Dopplergrams, and continuum images of the Sun in the Fe I 6173A line. We determine the RV modulation arising from the suppression of granular blueshift in magnetised regions and the flux imbalance induced by dark spots and bright faculae. The rms velocity amplitudes of these contributions are 2.40 m/s and 0.41 m/s, respectively, which confirms that the inhibition of convection is the dominant source of activity-induced RV variations at play, in accordance with previous studies. We find the Doppler imbalances of spot and plage regions to be only weakly anticorrelated. Lightcurves can thus only give incomplete predictions of convective blueshift suppression. We must instead seek proxies that track the plage coverage on the visible stellar hemisphere directly. The chromospheric flux index R_HK derived from the HARPS spectra performs poorly in this respect, possibly because of the differences in limb brightening/darkening in the chromosphere and photosphere. We also find that the activity-driven RV variations of the Sun are strongly correlated with its full-disc magnetic flux density, which may become a useful proxy for activity-related RV noise.