The atomic structure of the supercooled liquid has often been discussed as a key source of glass formation in metals. The presence of icosahedrally-coordinated clusters and their tendency to form networks have been identified as one possible structural trait leading to glass forming ability in the Cu-Zr binary system. In this work, we show that this theory is insufficient to explain glass formation at all compositions in that binary system. Instead, we propose that the formation of ideally-packed clusters at the expense of atomic arrangements with excess or deficient free volume can explain glass-forming by a similar mechanism. We show that this behavior is reflected in the structural relaxation of a metallic glass during constant pressure cooling and the time evolution of structure at a constant volume. We then demonstrate that this theory is sufficient to explain slowed diffusivity in compositions across the range of Cu-Zr metallic glasses.