We report experiments on hard sphere colloidal glasses that reveal a type of shear banding hitherto unobserved in soft glasses. We present a scenario that relates this to an instability arising from shear-concentration coupling, a mechanism previously thought unimportant in this class of materials. Below a characteristic shear rate $dotgamma_c$ we observe increasingly non-linear velocity profiles and strongly localized flows. We attribute this trend to very slight concentration gradients (likely to evade direct detection) arising in the unstable flow regime. A simple model accounts for both the observed increase of $dotgamma_c$ with concentration, and the fluctuations observed in the flow.