Selenium and tellurium are among the few elements that form $AB$O$_3$ perovskite structures with a four valent ion in the $A$ site. This leads to highly distorted structures and unusual magnetic behavior. Here we investigate the Co and Ni selenite and tellurite compounds, CoSeO$_3$, CoTeO$_3$, NiSeO$_3$ and NiTeO$_3$ using first principles calculations. We find an interplay of crystal field and Jahn-Teller distortions that underpin the electronic and magnetic properties. While all compounds are predicted to show an insulating G-type antiferromagnetic ground state, there is a considerable difference in the anisotropy of the exchange interactions between the Ni and Co compounds. This is related to the Jahn-Teller distortion. Finally, we observe that these four compounds show characteristics generally associated with Mott insulators, even when described at the level of standard density functional theory. These are then dense bulk band or Slater, Mott-type insulators.