We provide the reader with an accessible yet rigorous introduction to Bayesian optimisation with Gaussian processes (BOGP) for the purpose of solving a wide variety of radio resource management (RRM) problems. We believe that BOGP is a powerful tool that has been somewhat overlooked in RRM research, although it elegantly addresses pressing requirements for fast convergence, safe exploration, and interpretability. BOGP also provides a natural way to exploit prior knowledge during optimization. After explaining the nuts and bolts of BOGP, we delve into more advanced topics, such as the choice of the acquisition function and the optimization of dynamic performance functions. Finally, we put the theory into practice for the RRM problem of uplink open-loop power control (OLPC) in 5G cellular networks, for which BOGP is able to converge to almost optimal solutions in tens of iterations without significant performance drops during exploration.