We present a combined experimental and theoretical study on the rotationally inelastic scattering of OH ($X,^2Pi_{3/2}, J=3/2, f$) radicals with the collision partners He, Ne, Ar, Kr, Xe, and D$_2$ as a function of the collision energy between $sim 70$ cm$^{-1}$ and 400~cm$^{-1}$. The OH radicals are state selected and velocity tuned prior to the collision using a Stark decelerator, and field-free parity-resolved state-to-state inelastic relative scattering cross sections are measured in a crossed molecular beam configuration. For all OH-rare gas atom systems excellent agreement is obtained with the cross sections predicted by close-coupling scattering calculations based on accurate emph{ab initio} potential energy surfaces. This series of experiments complements recent studies on the scattering of OH radicals with Xe [Gilijamse emph{et al.}, Science {bf 313}, 1617 (2006)], Ar [Scharfenberg emph{et al.}, Phys. Chem. Chem. Phys. {bf 12}, 10660 (2010)], He, and D$_2$ [Kirste emph{et al.}, Phys. Rev. A {bf 82}, 042717 (2010)]. A comparison of the relative scattering cross sections for this set of collision partners reveals interesting trends in the scattering behavior.