Quasar absorption spectral data indicate the presence of a spatial gradient in the electromagnetic fine-structure constant $alpha$ on cosmological length scales. We point out that experiments with accelerometers, including torsion pendula and atom interferometers, can be used as sensitive probes of cosmological spatial gradients in the fundamental constants of nature, which give rise to equivalence-principle-violating forces on test masses. Using laboratory data from the Eot-Wash experiment, we constrain spatial gradients in $alpha$ along any direction to be $| boldsymbol{ abla} alpha / alpha | < 6.6 times 10^{-4}~(textrm{Glyr})^{-1}$ at $95%$ confidence level. Our result represents an order of magnitude improvement over laboratory bounds from clock-based searches for a spatial gradient in $alpha$ directed along the observed cosmological $alpha$-dipole axis. Improvements to accelerometer experiments in the foreseeable future are expected to provide sufficient sensitivity to test the cosmological $alpha$-dipole seen in astrophysical data.