We discuss a numerical model for black hole growth and its associated feedback processes that for the first time allows cosmological simulations of structure formation to self-consistently follow the build up of the cosmic population of galaxies and active galactic nuclei. Our model assumes that seed black holes are present at early cosmic epochs at the centres of forming halos. We then track their growth from gas accretion and mergers with other black holes in the course of cosmic time. For black holes that are active, we distinguish between two distinct modes of feedback, depending on the black hole accretion rate itself. Black holes that accrete at high rates are assumed to be in a `quasar regime, where we model their feedback by thermally coupling a small fraction of their bolometric luminosity to the surrounding gas. For black holes with low accretion rates, we conjecture that most of their feedback occurs in mechanical form, where AGN-driven bubbles are injected into a gaseous environment. Using our new model, we carry out TreeSPH cosmological simulations on the scales of individual galaxies to those of massive galaxy clusters, both for isolated systems and for cosmological boxes. We demonstrate that our model produces results for the black hole and stellar mass densities in broad agreement with observational constraints. We find that the black holes significantly influence the evolution of their host galaxies, changing their star formation history, their amount of cold gas, and their colours. Also, the properties of intracluster gas are affected strongly by the presence of massive black holes in the cores of galaxy clusters, leading to shallower metallicity and entropy profiles, and to a suppression of strong cooling flows. [Abridged]