The negatively-charged silicon-vacancy (SiV$^-$) center in diamond is a promising single photon source for quantum communications and information processing. However, the centers implementation in such quantum technologies is hindered by contention surrounding its fundamental properties. Here we present optical polarization measurements of single centers in bulk diamond that resolve this state of contention and establish that the center has a $langle111rangle$ aligned split-vacancy structure with $D_{3d}$ symmetry. Furthermore, we identify an additional electronic level and evidence for the presence of dynamic Jahn-Teller effects in the centers 738 nm optical resonance.