The surfaces generated by cleaving non-polar, two-dimensional oxides are often considered to be perfect or ideal. However, single particle spectroscopies on Sr2RuO4, an archetypal non-polar two dimensional oxide, show significant cleavage temperature dependence. We demonstrate that this is not a consequence of the intrinsic characteristics of the surface: lattice parameters and symmetries, step heights, atom positions, or density of states. Instead, we find a marked increase in the density of defects at the mesoscopic scale with increased cleave temperature. The potential generality of these defects to oxide surfaces may have broad consequences to interfacial control and the interpretation of surface sensitive measurements.