The cosmic black hole accretion density (BHAD) is critical for our understanding of the formation and evolution of supermassive black holes (BHs). However, at high redshifts ($z>3$), X-ray observations report BHADs significantly ($sim 10$ times) lower than those predicted by cosmological simulations. It is therefore paramount to constrain the high-$z$ BHAD using independent methods other than direct X-ray detections. The recently established relation between star formation rate and BH accretion rate among bulge-dominated galaxies provides such a chance, as it enables an estimate of the BHAD from the star-formation histories (SFHs) of lower-redshift objects. Using the CANDELS Lyman-$alpha$ Emission At Reionization (CLEAR) survey, we model the SFHs for a sample of 108 bulge-dominated galaxies at $z=$0.7-1.5, and further estimate the BHAD contributed by their high-$z$ progenitors. The predicted BHAD at $zapprox 4$-5 is consistent with the simulation-predicted values, but higher than the X-ray measurements (by $approx$3-10 times at $z=$4-5). Our result suggests that the current X-ray surveys could be missing many heavily obscured Compton-thick active galactic nuclei (AGNs) at high redshifts. However, this BHAD estimation assumes that the high-$z$ progenitors of our $z=$0.7-1.5 sample remain bulge-dominated where star formation is correlated with BH cold-gas accretion. Alternatively, our prediction could signify a stark decline in the fraction of bulges in high-$z$ galaxies (with an associated drop in BH accretion). JWST and Origins will resolve the discrepancy between our predicted BHAD and the X-ray results by constraining Compton-thick AGN and bulge evolution at high redshifts.