The neutral charge state plays an important role in quantum information and sensing applications based on nitrogen-vacancy centers. However, the orbital and spin dynamics remain unexplored. Here, we use resonant excitation of single centers to directly reveal the fine structure, enabling selective addressing of spin-orbit states. Through pump-probe experiments, we find the orbital relaxation time (430ns at 4.7K) and measure its temperature-dependence up to 11.8K. Finally we reveal the spin relaxation time (1.5s), and realize projective high-fidelity single-shot readout of the spin state ($geq98%$).