We study the Intra-Halo Stellar Component (IHSC) of Milky Way-mass systems up to galaxy clusters in the Horizon-AGN cosmological hydrodynamical simulation. We identify the IHSC using an improved phase-space galaxy finder algorithm which provides an adaptive, physically motivated and shape-independent definition of this stellar component, that can be applied to halos of arbitrary masses. We explore the IHSC mass fraction-total halos stellar mass, $f_{M*,IHSC}-M*$, relation and the physical drivers of its scatter. We find that on average the $f_{M*,IHSC}$ increases with $M_{*,tot}$, with the scatter decreasing strongly with mass from 2 dex at $M_{*,tot}sim10^{11}M_odot$ to 0.3 dex at group masses. At high masses, $M_{*,tot}>10^{11.5}M_odot$, $f_{M*,IHSC}$ increases with the number of substructures, and with the mass ratio between the central galaxy and largest satellite, at fixed $M_{*,tot}$. From mid-size groups and systems below $M_{*,tot}<10^{12}M_odot$, we find that the central galaxys stellar rotation-to-dispersion velocity ratio, V/{sigma}, displays the strongest (anti)-correlation with $f_{M*,IHSC}$ at fixed $M_{*,tot}$ of all the galaxy and halo properties explored, transitioning from $f_{M*,IHSC}$<0.1% for high V/{sigma}, to $f_{M*,IHSC}sim5$% for low V/{sigma} galaxies. By studying the $f_{M*,IHSC}$ temporal evolution, we find that, in the former, mergers not always take place, but if they did, they happened early (z>1), while the high $f_{M*,IHSC}$ population displays a much more active merger history. In the case of massive groups and galaxy clusters, $M_{*,tot}>10^{12}M_odot$, a fraction $f_{M*,IHSC}sim$10-20% is reached at $zsim1$ and then they evolve across lines of constant $f_{M*,IHSC}$ modulo some small perturbations. Because of the limited simulations volume, the latter is only tentative and requires a larger sample of simulated galaxy clusters to confirm.