The characteristic microstructure of Sm(Co,Fe,Cu,Zr)$_z$ alloys with SmCo$_5$ cell walls in Sm$_2$Co$_{17}$ cells, all intersected by Zr-rich platelets, makes them some of the best performing high-temperature permanent magnets. Plentiful research has been performed to tailor the microstructure at the nanoscale, but due to its complexity many questions remain unanswered about the effect of the individual phases on the magnetic performance at different temperatures. Here, we explore this mechanism effect for three different Sm(Co,Fe,Cu,Zr)$_z$ alloys by deploying high-resolution magnetic imaging via in-situ transmission electron microscopy and three-dimensional chemical analysis using atom probe tomography. We show that their microstructures differ in terms of SmCo$_5$ cell-wall and Z-phase size and density, as well as the Cu concentration in the cell walls, and demonstrate how these features influence the magnetic domain size and density and thus form different magnetic textures. Moreover, we illustrate that the dominant coercivity mechanism at room temperature is domain-wall pinning and show that magnets with a denser cell-wall network, a steeper Cu gradient across the cell-wall boundary, and thinner Z-phase platelets have a higher coercivity. We also show that the coercivity mechanism at high temperatures is domain-wall nucleation at the cell walls. Increasing the Cu concentration inside the cell walls decreases the transition temperature between pinning and nucleation, significantly decreasing the coercivity with increasing temperature. We therefore provide a detailed explanation of how the microstructure on the atomic to nanoscale directly affects the magnetic performance and provide detailed guidelines for an improved design of Sm(Co,Fe,Cu,Zr)$_z$ magnets.