We demonstrate that cation-related localized states strongly perturb the band structure of $text{Al}_{1-x}text{In}_x$N leading to a strong band gap bowing at low In content. Our first-principles calculations show that In-related localized states are formed both in the conduction and the valence band in $text{Al}_{1-x}text{In}_x$N for low In composition, $x$, and that these localized states dominate the evolution of the band structure with increasing $x$. Therefore, the commonly used assumption of a single composition-independent bowing parameter breaks down when describing the evolution both of the conduction and of the valence band edge in $text{Al}_{1-x}text{In}_x$N.