A coupled atomistic spin and lattice dynamics approach is developed which merges the dynamics of these two degrees of freedom into a single set of coupled equations of motion. The underlying microscopic model comprises local exchange interactions between the electron spin and magnetic moment and the local couplings between the electronic charge and lattice displacements. An effective action for the spin and lattice variables is constructed in which the interactions among the spin and lattice components are determined by the underlying electronic structure. In this way, expressions are obtained for the electronically mediated couplings between the spin and lattice degrees of freedom, besides the well known inter-atomic force constants and spin-spin interactions. These former susceptibilities provide an atomistic ab initio description for the coupled spin and lattice dynamics. It is important to notice that this theory is strictly bilinear in the spin and lattice variables and provides a minimal model for the coupled dynamics of these subsystems and that the two subsystems are treated on the same footing. Questions concerning time-reversal and inversion symmetry are rigorously addressed and it is shown how these aspects are absorbed in the tensor structure of the interaction fields. By means of these results regarding the spin-lattice coupling, simple explanations of ionic dimerization in double anti-ferromagnetic materials, as well as, charge density waves induced by a non-uniform spin structure are given. In the final parts, a set of coupled equations of motion for the combined spin and lattice dynamics are constructed, which subsequently can be reduced to a form which is analogous to the Landau-Lifshitz-Gilbert equations for spin dynamics and damped driven mechanical oscillator for the ...