We present a calculation of the hadronic vacuum polarization contribution to the muon anomalous magnetic moment, $a_mu^{mathrm hvp}$, in lattice QCD employing dynamical up and down quarks. We focus on controlling the infrared regime of the vacuum polarization function. To this end we employ several complementary approaches, including Pade fits, time moments and the time-momentum representation. We correct our results for finite-volume effects by combining the Gounaris-Sakurai parameterization of the timelike pion form factor with the Luscher formalism. On a subset of our ensembles we have derived an upper bound on the magnitude of quark-disconnected diagrams and found that they decrease the estimate for $a_mu^{mathrm hvp}$ by at most 2%. Our final result is $a_mu^{mathrm hvp}=(654pm32,{}^{+21}_{-23})cdot 10^{-10}$, where the first error is statistical, and the second denotes the combined systematic uncertainty. Based on our findings we discuss the prospects for determining $a_mu^{mathrm hvp}$ with sub-percent precision.