Gauge theories are the cornerstone of our understanding of fundamental interactions among particles. Their properties are often probed in dynamical experiments, such as those performed at ion colliders and high-intensity laser facilities. Describing the evolution of these strongly coupled systems is a formidable challenge for classical computers, and represents one of the key open quests for quantum simulation approaches to particle physics phenomena. Here, we show how recent experiments done on Rydberg atom chains naturally realize the real-time dynamics of a lattice gauge theory at system sizes at the boundary of classical computational methods. We prove that the constrained Hamiltonian dynamics induced by strong Rydberg interactions maps exactly onto the one of a $U(1)$ lattice gauge theory. Building on this correspondence, we show that the recently observed anomalously slow dynamics corresponds to a string-inversion mechanism, reminiscent of the string-breaking typically observed in gauge theories. This underlies the generality of this slow dynamics, which we illustrate in the context of one-dimensional quantum electrodynamics on the lattice. Within the same platform, we propose a set of experiments that generically show long-lived oscillations, including the evolution of particle-antiparticle pairs. Our work shows that the state of the art for quantum simulation of lattice gauge theories is at 51 qubits, and connects the recently observed slow dynamics in atomic systems to archetypal phenomena in particle physics