For groups of diffeomorphisms of $T^2$ containing an Anosov diffeomorphism, we give a complete classification for polycyclic Abelian-by-Cyclic group actions on $T^2$ up to both topological conjugacy and smooth conjugacy under mild assumptions. Along the way, we also prove a Tits alternative type theorem for some groups of diffeomorphisms of $T^2$.