Bismuth oxyselenide (Bi$_2$O$_2$Se) attracts great interest as a potential n-type complement to p-type thermoelectric oxides in practical applications. Previous investigations were generally focused on polycrystals. Here, we performed a study on the thermoelectric properties of Bi$_2$O$_2$Se single crystals. Our samples exhibit electron mobility as high as 250 cm$^2.$V$^{-1}$.s$^{-1}$ and thermal conductivity as low as $2$ W.m$^{-1}$.K$^{-1}$ near room temperature. The maximized figure of merit is yielded to be 0.188 at 390 K, higher than that of polycrystals. Consequently, a rough estimation of the phonon mean free path ($ell_textrm{ph}$) from the kinetic model amounts to 12 $r{A}$ at 390 K and follows a $T^{-1}$ behavior. An extrapolation of $ell_textrm{ph}$ to higher temperatures indicates that this system approaches the Ioffe-Regel limit at about 1100 K. In light of the phonon dispersions, we argue that the ultralow $ell_textrm{ph}$ is attributed to intense anharmonic phonon-phonon scattering, including Umklapp process and acoustic to optical phonon scattering. Our results suggest that single crystals provide a further improvement of thermoelectric performance of Bi$_2$O$_2$Se.