Diamond and cBN are two of the most promising ultra-wide-band-gap (UWBG) semiconductors for applications in high-power high-frequency electronic devices. Yet despite extensive studies on carrier transport in these materials, there are large discrepancies in their reported carrier mobilities. In this work, we investigate the phonon- and dopant-limited electron and hole mobility of cBN and diamond with atomistic first-principles calculations in order to understand their fundamental upper bounds to carrier transport. Our results show that although the phonon-limited electron mobilities are comparable between cBN and diamond, the hole mobility is significantly lower in cBN due to its heavier hole effective mass. Moreover, although lattice scattering dominates the mobility at low doping, neutral impurity scattering becomes the dominant scattering mechanism at higher dopant concentrations due to the high dopant ionization energies. Our analysis provides critical insights and reveals the intrinsic upper limits to the carrier mobilities of diamond and cBN as a function of doping and temperature for applications in high-power electronic devices.