We obtain a microscopic description of the interaction between electron spins in bulk semiconductors and in pairs of semiconductor quantum dots. Treating the k.p band mixing and the Coulomb interaction on the same footing, we obtain in the third order an asymmetric contribution to the exchange interaction arising from the coupling between the spin of one electron and the relative orbital motion of the other. This contribution does not depend on the inversion asymmetry of the crystal. We find that it is ~0.001 of the isotropic exchange, which is of interest in quantum information. Detailed evaluations are given for several quantum dot systems.