We present a mechanism to protect quantum information stored in an ensemble of nuclear spins in a semiconductor quantum dot. When the dot is charged the nuclei interact with the spin of the excess electron through the hyperfine coupling. If this coupling is made off-resonant it leads to an energy gap between the collective storage states and all other states. We show that the energy gap protects the quantum memory from local spin-flip and spin-dephasing noise. Effects of non-perfect initial spin polarization and inhomogeneous hyperfine coupling are discussed.