Understanding cities is central to addressing major global challenges from climate and health to economic resilience. Although increasingly perceived as fundamental socio-economic units, the detailed fabric of urban economic activities is only now accessible to comprehensive analyses with the availability of large datasets. Here, we study abundances of business categories across U.S. metropolitan statistical areas to investigate how diversity of economic activities depends on city size. A universal structure common to all cities is revealed, manifesting self-similarity in internal economic structure as well as aggregated metrics (GDP, patents, crime). A derivation is presented that explains universality and the observed empirical distribution. The model incorporates a generalized preferential attachment process with ceaseless introduction of new business types. Combined with scaling analyses for individual categories, the theory quantitatively predicts how individual business types systematically change rank with city size, thereby providing a quantitative means for estimating their expected abundances as a function of city size. These results shed light on processes of economic differentiation with scale, suggesting a general structure for the growth of national economies as integrated urban systems.