This paper introduces a theoretical framework for understanding the accumulation of non-Abelian geometric phases in rotating nitrogen-vacancy centers in diamond. Specifically, we consider how degenerate states can be achieved and demonstrate that the resulting geometric phase for multiple paths is non-Abelian. We find that the non-Abelian nature of the phase is robust to fluctuations in the path and magnetic field. In contrast to previous studies of the accumulation of Abelian geometric phases for nitrogen-vacancy centers under rotation we find that the limiting time-scale is $T_{1}$. As such a non-Abelian geometric phase accumulation in nitrogen-vacancy centers has potential advantages for applications as gyroscopes.