We investigate how structural relaxation in mixtures with strong dynamical asymmetry is affected by the microscopic dynamics. Brownian and Newtonian dynamics simulations of dense mixtures of fast and slow hard spheres reveal a striking trend reversal. Below a critical density, increasing the mobility of the fast particles fluidizes the system, yet, above that critical density, the same increase in mobility strongly hinders the relaxation of the slow particles. The critical density itself does not depend on the dynamical asymmetry and can be identified with the glass-transition density of the mode-coupling theory. The asymptotic dynamics close to the critical density is universal, but strong pre-asymptotic effects prevail in mixtures with additional size asymmetry. This observation reconciles earlier findings of a strong dependence on kinetic parameters of glassy dynamics in colloid--polymer mixtures with the paradigm that the glass transition is determined by the properties of configuration space alone.