We establish lower bounds on the dimensions in which arithmetic groups with torsion can act on acyclic manifolds and homology spheres. The bounds rely on the existence of elementary p-groups in the groups concerned. In some cases, including Sp(2n,Z), the bounds we obtain are sharp: if X is a generalized Z/3-homology sphere of dimension less than 2n-1 or a Z/3-acyclic Z/3-homology manifold of dimension less than 2n, and if n geq 3, then any action of Sp(2n,Z) by homeomorphisms on X is trivial; if n = 2, then every action of Sp(2n,Z) on X factors through the abelianization of Sp(4,Z), which is Z/2.