At the highest redshifts, z>6, several tens of luminous quasars have been detected. The search for fainter AGN, in deep X-ray surveys, has proven less successful, with few candidates to date. An extrapolation of the relationship between black hole (BH) and bulge mass would predict that the sample of z>6 galaxies host relatively massive BHs (>1e6 Msun), if one assumes that total stellar mass is a good proxy for bulge mass. At least a few of these BHs should be luminous enough to be detectable in the 4Ms CDFS. The relation between BH and stellar mass defined by local moderate-luminosity AGN in low-mass galaxies, however, has a normalization that is lower by approximately an order of magnitude compared to the BH-bulge mass relation. We explore how this scaling changes the interpretation of AGN in the high-z Universe. Despite large uncertainties, driven by those in the stellar mass function, and in the extrapolation of local relations, one can explain the current non-detection of moderate-luminosity AGN in Lyman Break Galaxies if galaxies below 1e11 Msun are characterized by the low-normalization scaling, and, even more so, if their Eddington ratio is also typical of moderate-luminosity AGN rather than luminous quasars. AGN being missed by X-ray searches due to obscuration or instrinsic X-ray weakness also remain a possibility.