We propose a method to obtain optimal protocols for adiabatic ground-state preparation near the adiabatic limit, extending earlier ideas from [D. A. Sivak and G. E. Crooks, Phys. Rev. Lett. 108, 190602 (2012)] to quantum non-dissipative systems. The space of controllable parameters of isolated quantum many-body systems is endowed with a Riemannian quantum metric structure, which can be exploited when such systems are driven adiabatically. Here, we use this metric structure to construct optimal protocols in order to accomplish the task of adiabatic ground-state preparation in a fixed amount of time. Such optimal protocols are shown to be geodesics on the parameter manifold, maximizing the local fidelity. Physically, such protocols minimize the average energy fluctuations along the path. Our findings are illustrated on the Landau-Zener model and the anisotropic XY spin chain. In both cases we show that geodesic protocols drastically improve the final fidelity. Moreover, this happens even if one crosses a critical point, where the adiabatic perturbation theory fails.