Fully accounting for non-dipole effects in the electron dynamics, double ionization is studied for He driven by a near-infrared laser field and for Xe driven by a mid-infrared laser field. Using a three-dimensional semiclassical model, the average sum of the electron momenta along the propagation direction of the laser field is computed. This sum is found to be an order of magnitude larger than twice the average electron momentum along the propagation direction of the laser field in single ionization. Moreover, the average sum of the electron momenta in double ionization is found to be maximum at intensities smaller than the intensities satisfying previously predicted criteria for the onset of magnetic field effects. It is shown that strong recollisions are the reason for this unexpectedly large value of the sum of the momenta along the direction of the magnetic component of the Lorentz force.