We compute the shear viscosity of QCD with matter, including almost all next-to-leading order corrections -- that is, corrections suppressed by one power of $g$ relative to leading order. We argue that the still missing terms are small. The next-to-leading order corrections are large and bring $eta/s$ down by more than a factor of 3 at physically relevant couplings. The perturbative expansion is problematic even at $T simeq 100$ GeV. The largest next-to-leading order correction to $eta/s$ arises from modifications to the qhat parameter, which determines the rate of transverse momentum diffusion. We also explore quark number diffusion, and shear viscosity in pure-glue QCD and in QED.