The climate system is a complex, chaotic system with many degrees of freedom and variability on a vast range of temporal and spatial scales. Attaining a deeper level of understanding of its dynamical processes is a scientific challenge of great urgency, especially given the ongoing climate change and the evolving climate crisis. In statistical physics, complex, many-particle systems are studied successfully using the mathematical framework of Large Deviation Theory (LDT). A great potential exists for applying LDT to problems relevant for geophysical fluid dynamics and climate science. In particular, LDT allows for understanding the fundamental properties of persistent deviations of climatic fields from the long-term averages and for associating them to low-frequency, large scale patterns of climatic variability. Additionally, LDT can be used in conjunction with so-called rare events algorithms to explore rarely visited regions of the phase space and thus to study special dynamical configurations of the climate. These applications are of key importance to improve our understanding of high-impact weather and climate events. Furthermore, LDT provides powerful tools for evaluating the probability of noise-induced transitions between competing metastable states of the climate system or of its components. This in turn essential for improving our understanding of the global stability properties of the climate system and of its predictability of the second kind in the sense of Lorenz. The goal of this review is manifold. First, we want to provide an introduction to the derivation of large deviation laws in the context of stochastic processes. We then relate such results to the existing literature showing the current status of applications of LDT in climate science and geophysical fluid dynamics. Finally, we propose some possible lines of future investigations.