Crystal electric field states in rare earth intermetallics show an intricate entanglement with the many-body physics that occurs in these systems and that is known to lead to a plethora of electronic phases. Here, we attempt to trace different contributions to the crystal electric field (CEF) splittings in CeIrIn$_5$, a heavy-fermion compound and member of the Ce$M$In$_5$ ($M$= Co, Rh, Ir) family. To this end, we utilize high-resolution resonant angle-resolved photoemission spectroscopy (ARPES) and present a spectroscopic study of the electronic structure of this unconventional superconductor over a wide temperature range. As a result, we show how ARPES can be used in combination with thermodynamic measurements or neutron scattering to disentangle different contributions to the CEF splitting in rare earth intermetallics. We also find that the hybridization is stronger in CeIrIn$_5$ than CeCoIn$_5$ and the effects of the hybridization on the Fermi volume increase is much smaller than predicted. By providing the first experimental evidence for $4f_{7/2}^{1}$ splittings which, in CeIrIn$_5$, split the octet into four doublets, we clearly demonstrate the many-body origin of the so-called $4f_{7/2}^{1}$ state.