We show how U(1) lattice gauge theories display key signatures of ergodicity breaking in the presence of a random charge background. Contrary to the widely studied case of spin models, in the presence of Coulomb interactions, the spectral properties of such lattice gauge theories are very weakly affected by finite-volume effects. This allows to draw a sharp boundary for the ergodic regime, and thus the breakdown of quantum chaos for sufficiently strong gauge couplings, at the system sizes accessible via exact diagonalization. Our conclusions are independent on the value of a background topological angle, and are contrasted with a gauge theory with truncated Hilbert space, where instead we observe very strong finite-volume effects akin to those observed in spin chains.