We compute the non-planar contribution to the universal anomalous dimension of twist-two operators in N=4 supersymmetric Yang-Mills theory at four loops through Lorentz spin eighteen. Exploiting the results of this and our previous calculations along with recent analytic results for the cusp anomalous dimension and some expected analytic properties, we reconstruct a general expression valid for arbitrary Lorentz spin. We study various properties of this general result, such as its large-spin limit, its small-x limit, and others. In particular, we present a prediction for the non-planar contribution to the anomalous dimension of the single-magnon operator in the beta-deformed version of the theory.