The sensitivity of X-ray facilities will increase with the upcoming Athena and the AXIS and Lynx concept missions. These new instruments will allow us to detect fainter active galactic nuclei (AGN), therefore increasing our understanding of the supermassive black hole (BH) population in a luminosity regime that can be dominated by X-ray binaries. We analyze the population of faint AGN (L_x (2-10 keV) < 10^42 erg/s) in the Illustris, TNG100, EAGLE, and SIMBA large-scale cosmological simulations. We find that the properties of the faint AGN host galaxies vary from simulation to simulation. In Illustris and EAGLE, faint AGN of L_x (2-10 keV) ~ 10^38 erg/s are powered by low-mass BHs and they are typically located in low-mass star-forming galaxies. In TNG100 and SIMBA, they are mostly associated with more massive BHs in quenched massive galaxies. By modeling the X-ray binary populations (XRB) of the simulated galaxies using empirical scaling relations, we demonstrate that while the AGN dominate the hard X-ray galaxy luminosity at high redshift (z>2), the X-ray binaries often dominate at low redshift (z<2). The X-ray luminosity of star-forming galaxies is often dominated by AGN emission, and of quenched galaxies by XRB emission. These differences can be used to discriminate between galaxy formation models with future high-resolution X-ray observations. To pave the way, we compare the total AGN+XRB hard X-ray luminosity of simulated faint AGN host galaxies to observations of stacked galaxies from Chandra. In general, our comparison indicates that the simulations post-processed with our X-ray modeling assumptions tend to overestimate the total AGN+XRB X-ray luminosity. We find that AGN obscuration can affect by almost one order of magnitude the median AGN+XRB luminosity. Some simulations reveal clear AGN trends as a function of stellar mass, which are less apparent in the current observations.