Magnetic fields are everywhere in nature and they play an important role in every astronomical environment which involves the formation of plasma and currents. It is natural therefore to suppose that magnetic fields could be present in the turbulent high temperature environment of the big bang. Such a primordial magnetic field (PMF) would be expected to manifest itself in the cosmic microwave background (CMB) temperature and polarization anisotropies, and also in the formation of large- scale structure. In this review we summarize the theoretical framework which we have developed to calculate the PMF power spectrum to high precision. Using this formulation, we summarize calculations of the effects of a PMF which take accurate quantitative account of the time evolution of the cut off scale. We review the constructed numerical program, which is without approximation, and an improvement over the approach used in a number of previous works for studying the effect of the PMF on the cosmological perturbations. We demonstrate how the PMF is an important cosmological physical process on small scales. We also summarize the current constraints on the PMF amplitude $B_lambda$ and the power spectral index $n_B$ which have been deduced from the available CMB observational data by using our computational framework.