Turbulence is omnipresent in Nature and technology, governing the transport of heat, mass, and momentum on multiple scales. For real-world applications of wall-bounded turbulence, the underlying surfaces are virtually always rough; yet characterizing and understanding the effects of wall roughness for turbulence remains a challenge, especially for rotating and thermally driven turbulence. By combining extensive experiments and numerical simulations, here, taking as example the paradigmatic Taylor-Couette system (the closed flow between two independently rotating coaxial cylinders), we show how wall roughness greatly enhances the overall transport properties and the corresponding scaling exponents. If only one of the walls is rough, we reveal that the bulk velocity is slaved to the rough side, due to the much stronger coupling to that wall by the detaching flow structures. If both walls are rough, the viscosity dependence is thoroughly eliminated in the boundary layers and we thus achieve asymptotic ultimate turbulence, i.e. the upper limit of transport, whose existence had been predicted by Robert Kraichnan in 1962 (Phys. Fluids {bf 5}, 1374 (1962)) and in which the scalings laws can be extrapolated to arbitrarily large Reynolds numbers.