The II-IV-N$_2$ class of heterovalent ternary nitrides have gained significant interest as alternatives to the III-nitrides for electronic and optoelectronic applications. In this study, we apply first-principles calculations based on density functional theory to systematically investigate the effects of structural distortions due to cation size mismatch on the configurational disorder of the cation sublattice and the valence band structure in this class of materials. We find that larger size mismatch between the group-II and the group-IV cations results in stronger lattice distortions from the ideal hexagonal ratio, which in turn inhibits the propensity of these materials towards octet-rule violating cation disorder. We also demonstrate that the formation energy of a single cation antisite pair, which is fast and simple to calculate, is a strong indicator of a materials propensity towards disorder. Furthermore, the breaking of in-plane symmetry leads to a splitting of the top three valence bands at $Gamma$, which is also directly related to the magnitude of structural distortions. Our work demonstrates that the structural and functional properties of the II-IV-N$_2$ materials can be finely tuned through controllable structural distortions that stem from the choice of cations.