We show that two different tensors defining the same translational invariant injective Projected Entangled Pair State (PEPS) in a square lattice must be the same up to a trivial gauge freedom. This allows us to characterize the existence of any local
or spatial symmetry in the state. As an application of these results we prove that a SU(2) invariant PEPS with half-integer spin cannot be injective, which can be seen as a Lieb-Shultz-Mattis theorem in this context. We also give the natural generalization for U(1) symmetry in the spirit of Oshikawa-Yamanaka-Affleck, and show that a PEPS with Wilson loops cannot be injective.
