Protons and neutrons in the atomic nucleus move in shells analogous to the electronic shell structures of atoms. Nuclear shell structure varies across the nuclear landscape due to changes of the nuclear mean field with the number of neutrons $N$ and protons $Z$. These variations can be probed with mass differences. The $N=Z=40$ self-conjugate nucleus $^{80}$Zr is of particular interest as its proton and neutron shell structures are expected to be very similar, and its ground state is highly deformed. In this work, we provide evidence for the existence of a deformed double shell closure in $^{80}$Zr through high precision Penning trap mass measurements of $^{80-83}$Zr. Our new mass values show that $^{80}$Zr is significantly lighter, and thus more bound than previously determined. This can be attributed to the deformed shell closure at $N=Z=40$ and the large Wigner energy. Our statistical Bayesian model mixing analysis employing several global nuclear mass models demonstrates difficulties with reproducing the observed mass anomaly using current theory.