The promise enabled by BAs high thermal conductivity in power electronics cannot be assessed without taking into account the reduction incurred when doping the material. Using first principles calculations, we determine the thermal conductivity reduction induced by different group IV impurities in BAs as a function of concentration and charge state. We unveil a general trend, where neutral impurities scatter phonons more strongly than the charged ones. $text{C}_{text{B}}$ and $text{Ge}_{text{As}}$ impurities show by far the weakest phonon scattering and retain BAs $kappa$ values of over $sim$ 1000 $text{W}cdottext{K}^{-1}cdottext{m}^{-1}$ even up to high densities making them ideal n-type and p-type dopants. Furthermore, going beyond the doping compensation threshold associated to Fermi level pinning triggers observable changes in the thermal conductivity. This informs design considerations on the doping of BAs, and it also suggests a direct way to determine the onset of compensation doping in experimental samples.