We present a density functional study of graphene adhesion on a realistic SiO$_2$ surface taking into account van der Waals (vdW) interactions. The SiO$_2$ substrate is modeled at the local scale by using two main types of surface defects, typical for amorphous silica: the oxygen dangling bond and three-coordinated silicon. The results show that the nature of adhesion between graphene and its substrate is qualitatively dependent on the surface defect type. In particular, the interaction between graphene and silicon-terminated SiO$_2$ originates exclusively from the vdW interaction, whereas the oxygen-terminated surface provides additional ionic contribution to the binding arising from interfacial charge transfer ($p$-type doping of graphene). Strong doping contrast for the different surface terminations provides a mechanism for the charge inhomogeneity of graphene on amorphous SiO$_2$ observed in experiments. We found that independent of the considered surface morphologies, the typical electronic structure of graphene in the vicinity of the Dirac point remains unaltered in contact with the SiO$_2$ substrate, which points to the absence of the covalent interactions between graphene and amorphous silica. The case of hydrogen-passivated SiO$_2$ surfaces is also examined. In this situation, the binding with graphene is practically independent of the type of surface defects and arises, as expected, from the vdW interactions. Finally, the interface distances obtained are shown to be in good agreement with recent experimental studies.