High-resolution bunched-beam collinear laser spectroscopy was used to measure the optical hyperfine spectra of the $^{43-51}$Ca isotopes. The ground state magnetic moments of $^{49,51}$Ca and quadrupole moments of $^{47,49,51}$Ca were measured for the first time, and the $^{51}$Ca ground state spin $I=3/2$ was determined in a model-independent way. Our results provide a critical test of modern nuclear theories based on shell-model calculations using phenomenological as well as microscopic interactions. The results for the neutron-rich isotopes are in excellent agreement with predictions using interactions derived from chiral effective field theory including three-nucleon forces, while lighter isotopes illustrate the presence of particle-hole excitations of the $^{40}$Ca core in their ground state.