We investigate the spin dynamics driven by terahertz magnetic fields in epitaxial thin films of cobalt in its three crystalline phases. The terahertz magnetic field generates a torque on the magnetization which causes it to precess for about 1 ps, with a sub-picosecond temporal lag from the driving force. Then, the magnetization undergoes natural damped THz oscillations at a frequency characteristic of the crystalline phase. We describe the experimental observations solving the inertial Landau-Lifshitz-Gilbert equation. Using the results from the relativistic theory of magnetic inertia, we find that the angular momentum relaxation time $eta$ is the only material parameter needed to describe all the experimental evidence. Our experiments suggest a proportionality between $eta$ and the strength of the magneto-crystalline anisotropy.