We present the results of large scale molecular dynamics simulations aimed at understanding the origins of high friction coefficients in pure metals, and their concomitant reduction in alloys and composites. We utilize a series of targeted simulations to demonstrate that different slip mechanisms are active in the two systems, leading to differing frictional behavior. Specifically, we show that in pure metals, sliding occurs along the crystallographic slip planes, whereas in alloys shear is accommodated by grain boundaries. In pure metals, there is significant grain growth induced by the applied shear stress and the slip planes are commensurate contacts with high friction. However, the presence of dissimilar atoms in alloys suppresses grain growth and stabilizes grain boundaries, leading to low friction via grain boundary sliding.