We investigate the physics of dipolar bosons in a two dimensional optical lattice. It is known that due to the long-range character of dipole-dipole interaction, the ground state phase diagram of a gas of dipolar bosons in an optical lattice presents novel quantum phases, like checkerboard and supersolid phases. In this paper, we consider the properties of the system beyond its ground state, finding that it is characterised by a multitude of almost degenerate metastable states, often competing with the ground state. This makes dipolar bosons in a lattice similar to a disordered system and opens possibilities of using them for quantum memories.