We find a family of complex saddle-points at large N of the matrix model for the superconformal index of SU(N) N=4 super Yang-Mills theory on $S^3 times S^1$ with one chemical potential $tau$. The saddle-point configurations are labelled by points $(m,n)$ on the lattice $Lambda_tau= mathbb{Z} tau +mathbb{Z}$ with $text{gcd}(m,n)=1$. The eigenvalues at a given saddle are uniformly distributed along a string winding $(m,n)$ times along the $(A,B)$ cycles of the torus $mathbb{C}/Lambda_tau$. The action of the matrix model extended to the torus is closely related to the Bloch-Wigner elliptic dilogarithm, and the related Bloch formula allows us to calculate the action at the saddle-points in terms of real-analytic Eisenstein series. The actions of $(0,1)$ and $(1,0)$ agree with that of pure AdS$_5$ and the supersymmetric AdS$_5$ black hole, respectively. The black hole saddle dominates the canonical ensemble when $tau$ is close to the origin, and there are new saddles that dominate when $tau$ approaches rational points. The extension of the action in terms of modular forms leads to a simple treatment of the Cardy-like limit $tauto 0$.