We proposed utilizing a medium with a high optical depth (OD) and a Rydberg state of low principal quantum number, $n$, to create a weakly-interacting many-body system of Rydberg polaritons, based on the effect of electromagnetically induced transparency (EIT). We experimentally verified the mean field approach to weakly-interacting Rydberg polaritons, and observed the phase shift and attenuation induced by the dipole-dipole interaction (DDI). The DDI-induced phase shift or attenuation can be viewed as a consequence of the elastic or inelastic collisions among the Rydberg polaritons. Using a weakly-interacting system, we further observed that a larger DDI strength caused a width of the momentum distribution of Rydberg polaritons at the exit of the system to become notably smaller as compared with that at the entrance. In this study, we took $n =32$ and the atomic (or polariton) density of 5$times10^{10}$ (or 2$times10^{9}$) cm$^{-3}$. The observations demonstrate that the elastic collisions are sufficient to drive the thermalization process in this weakly-interacting many-body system. The combination of the $mu$s-long interaction time due to the high-OD EIT medium and the $mu$m$^2$-size collision cross section due to the DDI suggests a new and feasible platform for the Bose-Einstein condensation of the Rydberg polaritons.