First-principles density functional theory methods are used to investigate the structure, energetics, and vibrational motions of the neutral vacancy defect in diamond. The measured optical absorption spectrum demonstrates that the tetrahedral $T_d$ point group symmetry of pristine diamond is maintained when a vacancy defect is present. This is shown to arise from the presence of a dynamic Jahn-Teller distortion that is stabilised by large vibrational anharmonicity. Our calculations further demonstrate that the dynamic Jahn-Teller-distorted structure of $T_d$ symmetry is lower in energy than the static Jahn-Teller distorted tetragonal $D_{2d}$ vacancy defect, in agreement with experimental observations. The tetrahedral vacancy structure becomes more stable with respect to the tetragonal structure by increasing temperature. The large anharmonicity arises mainly from quartic vibrations, and is associated with a saddle point of the Born-Oppenheimer surface and a minimum in the free energy. This study demonstrates that the behaviour of Jahn-Teller distortions of point defects can be calculated accurately using anharmonic vibrational methods. Our work will open the way for first-principles treatments of dynamic Jahn-Teller systems in condensed matter.