Adsorbate engineering offers a seemingly simple approach to tailor spin-orbit interactions in atomically thin materials and thus to unlock the much sought-after topological insulating phases in two dimensions. However, the observation of an Anderson topological transition induced by heavy adatoms has proved extremely challenging despite substantial experimental efforts. Here, we present a multi-scale approach combining advanced first-principles methods and accurate single-electron descriptions of adatom-host interactions using graphene as a prototypical system. Our study reveals a surprisingly complex structure in the interactions mediated by random adatoms, including hitherto neglected hopping processes leading to strong valley mixing. We argue that the unexpected intervalley scattering strongly impacts the ground state at low adatom coverage, which would provide a compelling explanation for the absence of a topological gap in recent experimental reports. Our conjecture is confirmed by real-space Chern number calculations and large-scale quantum transport simulations in disordered samples. This resolves an important controversy and suggests that a detectable topological gap can be achieved by engineering the spatial range of spin-orbit interactions.