Massive black holes (BHs) are at once exotic and yet ubiquitous, residing in the centers of massive galaxies in the local Universe. Recent years have seen remarkable advances in our understanding of how these BHs form and grow over cosmic time, during which they are revealed as active galactic nuclei (AGN). However, despite decades of research, we still lack a coherent picture of the physical drivers of BH growth, the connection between the growth of BHs and their host galaxies, the role of large-scale environment on the fueling of BHs, and the impact of BH-driven outflows on the growth of galaxies. In this paper we review our progress in addressing these key issues, motivated by the science presented at the What Drives the Growth of Black Holes? workshop held at Durham on 26th-29th July 2010, and discuss how these questions may be tackled with current and future facilities.