The counting of the dimension of the space of $U(N) times U(N) times U(N)$ polynomial invariants of a complex $3$-index tensor as a function of degree $n$ is known in terms of a sum of squares of Kronecker coefficients. For $n le N$, the formula can be expressed in terms of a sum of symmetry factors of partitions of $n$ denoted $Z_3(n)$. We derive the large $n$ all-orders asymptotic formula for $ Z_3(n)$ making contact with high order results previously obtained numerically. The derivation relies on the dominance in the sum, of partitions with many parts of length $1$. The dominance of other small parts in restricted partition sums leads to related asymptotic results. The result for the $3$-index tensor observables gives the large $n$ asymptotic expansion for the counting of bipartite ribbon graphs with $n$ edges, and for the dimension of the associated Kronecker permutation centralizer algebra. We explain how the different terms in the asymptotics are associated with probability distributions over ribbon graphs. The large $n$ dominance of small parts also leads to conjectured formulae for the asymptotics of invariants for general $d$-index tensors. The coefficients of $ 1/n$ in these expansions involve Stirling numbers of the second kind along with restricted partition sums.