The study of the space density of bright AGNs at $z>4$ has been subject to extensive effort given its importance for the estimate of the cosmological ionizing emissivity and growth of supermassive black holes. In this context we have recently derived high space densities of AGNs at $zsim 4$ and $-25<M_{1450}<-23$ in the COSMOS field from a spectroscopically complete sample. In the present paper we attempt to extend the knowledge of the AGN space density at fainter magnitudes ($-22.5<M_{1450}<-18.5$) in the $4<z<6.1$ redshift interval by means of a multiwavelength sample of galaxies in the CANDELS GOODS-South, GOODS-North and EGS fields. We use an updated criterion to extract faint AGNs from a population of NIR (rest-frame UV) selected galaxies at photometric $z>4$ showing X-ray detection in deep Chandra images available for the three CANDELS fields. We have collected a photometric sample of 32 AGN candidates in the selected redshift interval, six of which having spectroscopic redshifts. Including our COSMOS sample as well as other bright QSO samples allows a first guess on the shape of the UV luminosity function at $zsim 4.5$. The resulting emissivity and photoionization rate appear consistent with that derived from the photoionization level of the intergalactic medium at $zsim 4.5$. An extrapolation to $zsim 5.6$ suggests an important AGN contribution to the IGM ionization if there are no significant changes in the shape of the UV luminosity function.