Using a three-dimensional semiclassical model, we study double ionization for strongly-driven He fully accounting for magnetic field effects. For linearly and slightly elliptically polarized laser fields, we show that recollisions and the magnetic field combined act as a gate. This gate favors more transverse - with respect to the electric field - initial momenta of the tunneling electron that are opposite to the propagation direction of the laser field. In the absence of non-dipole effects, the transverse initial momentum is symmetric with respect to zero. We find that this asymmetry in the transverse initial momentum gives rise to an asymmetry in a double ionization observable. Finally, we show that this asymmetry in the transverse initial momentum of the tunneling electron accounts for a recently-reported unexpectedly large average sum of the electron momenta parallel to the propagation direction of the laser field.