Gravitational wave observations of compact binaries allow us to test general relativity (and modifications thereof) in the strong and highly-dynamical field regime of gravity. Here we confront two extensions to general relativity, dynamical Chern-Simons and Einstein-dilaton-Gauss-Bonnet theories, against the gravitational wave sources from the GWTC-1 and GWTC-2 catalogs by the LIGO-Virgo Collaboration. By stacking the posterior of individual events, we strengthen the constraint on the square root of the coupling parameter in Einstein-dilaton-Gauss-Bonnet gravity to $sqrt{alpha_{rm tiny EdGB}} < 1.7$ km, but we are unable to place meaningful constraints on dynamical Chern-Simons gravity. Importantly, we also show that our bounds are robust to (i) the choice of general-relativity base waveform model, upon which we add modifications, (ii) unknown higher post-Newtonian order terms in the modifications to general relativity, (iii) the small-coupling approximation, and (iv) uncertainties on the nature of the constituent compact objects.