The masses of supermassive black holes at the centres of local galaxies appear to be tightly correlated with the mass and velocity dispersions of their galactic hosts. However, the local Mbh-Mstar relation inferred from dynamically measured inactive black holes is up to an order-of-magnitude higher than some estimates from active black holes, and recent work suggests that this discrepancy arises from selection bias on the sample of dynamical black hole mass measurements. In this work we combine X-ray measurements of the mean black hole accretion luminosity as a function of stellar mass and redshift with empirical models of galaxy stellar mass growth, integrating over time to predict the evolving Mbh-Mstar relation. The implied relation is nearly independent of redshift, indicating that stellar and black hole masses grow, on average, at similar rates. Matching the de-biased local Mbh-Mstar relation requires a mean radiative efficiency ~0.15, in line with theoretical expectations for accretion onto spinning black holes. However, matching the raw observed relation for inactive black holes requires a mean radiative efficiency around 0.02, far below theoretical expectations. This result provides independent evidence for selection bias in dynamically estimated black hole masses, a conclusion that is robust to uncertainties in bolometric corrections, obscured active black hole fractions, and kinetic accretion efficiency. For our fiducial assumptions, they favour moderate-to-rapid spins of typical supermassive black holes, to achieve a mean radiative efficiency ~0.12-0.20. Our approach has similarities to the classic Soltan analysis, but by using galaxy-based data instead of integrated quantities we are able to focus on regimes where observational uncertainties are minimized.