The similarity in mechanical properties of dense active matter and sheared amorphous solids has been noted in recent years without a rigorous examination of the underlying mechanism. We develop a mean-field model that predicts that their critical behavior should be equivalent in infinite dimensions, up to a rescaling factor that depends on the correlation length of the applied field. We test these predictions in 2d using a new numerical protocol, termed `athermal quasi-static random displacement, and find that these mean-field predictions are surprisingly accurate in low dimensions. We identify a general class of perturbations that smoothly interpolate between the uncorrelated localized forces that occur in the high-persistence limit of dense active matter, and system-spanning correlated displacements that occur under applied shear. These results suggest a universal framework for predicting flow, deformation, and failure in active and sheared disordered materials.