The mechanical response of naturally abundant amorphous solids such as gels, jammed grains, and biological tissues are not described by the conventional paradigm of broken symmetry that defines crystalline elasticity. In contrast, the response of such athermal solids are governed by local conditions of mechanical equilibrium, i.e., force and torque balance of its constituents. Here we show that these constraints have the mathematical structure of a generalized electromagnetism, where the electrostatic limit successfully captures the anisotropic elasticity of amorphous solids. The emergence of elasticity from local mechanical constraints offers a new paradigm for systems with no broken symmetry, analogous to emergent gauge theories of quantum spin liquids. Specifically, our $U(1)$ rank-2 symmetric tensor gauge theory of elasticity translates to the electromagnetism of fractonic phases of matter with the stress mapped to electric displacement and forces to vector charges. We corroborate our theoretical results with numerical simulations of soft frictionless disks in both two and three dimensions, and experiments on frictional disks in two dimensions. We also present experimental evidence indicating that force chains in granular media are sub-dimensional excitations of amorphous elasticity similar to fractons.