It is a long-standing important issue in heavy fermion physics whether $f$-electrons are itinerant or localized when the magnetic order occurs. Here we report the {it in situ} scanning tunneling microscopy observation of the electronic structure in epitaxial thin films of CeRhIn$_5$, a prototypical heavy fermion compound with antiferromagnetic ground state. The conductance spectra above the Neel temperature $T_N$ clearly resolve the energy gap due to the hybridization between local 4$f$ electrons and conduction bands as well as the crystal electric field excitations. These structures persist even below $T_N$. Moreover, an additional dip in the conductance spectra develops due to the antiferromagnetic order. These results provide direct evidence for the presence of itinerant heavy $f$-electrons participating in the Fermi surface even in the magnetically ordered state of CeRhIn$_5$.