We report a comprehensive de Haas--van Alphen (dHvA) study of the heavy-fermion material CeRhIn$_5$ in magnetic fields up to 70~T. Several dHvA frequencies gradually emerge at high fields as a result of magnetic breakdown. Among them is the thermodynamically important $beta_1$ branch, which has not been observed so far. Comparison of our angule-dependent dHvA spectra with those of the non-$4f$ compound LaRhIn$_5$ and with band-structure calculations evidences that the Ce $4f$ electrons in CeRhIn$_5$ remain localized over the whole field range. This rules out any significant Fermi-surface reconstruction, either at the suggested nematic phase transition at $B^{*}approx$ 30~T or at the putative quantum critical point at $B_c simeq$ 50~T. Our results rather demonstrate the robustness of the Fermi surface and the localized nature of the 4$f$ electrons inside and outside of the antiferromagnetic phase.