We have investigated the grain boundary scattering effect on the thermal transport behavior of uranium dioxide (UO$_2$). The polycrystalline samples having different grain-sizes (0.125, 1.8, and 7.2 $mu$m) have been prepared by spark plasma sintering technique and characterized by x-ray powder diffraction (XRD), scanning electron microscope (SEM), and Raman spectroscopy. The thermal transport properties (the thermal conductivity and thermoelectric power) have been measured in the temperature range 2-300~K and the results were analyzed in terms of various physical parameters contributing to the thermal conductivity in these materials in relation to grain-size. We show that thermal conductivity decreases systematically with lowering grain-size in the temperatures below 30 K, where the boundary scattering dominates the thermal transport. At higher temperatures more scattering processes are involved in the heat transport in these materials, making the analysis difficult. We determined the grain boundary Kapitza resistance that would result in the observed increase in thermal conductivity with grain size, and compared the value with Kapitza resistances calculated for UO$_2$ using molecular dynamics from the literature.