We calculate the leading-order hadronic correction to the anomalous magnetic moments of each of the three charged leptons in the Standard Model: the electron, muon and tau. Working in two-flavor lattice QCD, we address essentially all sources of systematic error: lattice artifacts, finite-size effects, quark-mass extrapolation, momentum extrapolation and disconnected diagrams. The most significant remaining systematic error, the exclusion of the strange and charm quark contributions, will be addressed in our four-flavor calculation. We achieve a statistical accuracy of 2% or better for the physical values for each of the three leptons and the systematic errors are at most comparable.