We report on the non-adiabatic offset of the initial electron momentum distribution in the plane of polarization upon single ionization of argon by strong field tunneling and show how to experimentally control the degree of non-adiabaticity. Two-color counter- and co-rotating fields (390 and 780 nm) are compared to show that the non-adiabatic offset strongly depends on the temporal evolution of the laser electric field. We introduce a simple method for the direct access to the non-adiabatic offset using two-color counter- and co-rotating fields. Further, for a single-color circularly polarized field at 780 nm we show that the radius of the experimentally observed donut-like distribution increases for increasing momentum in the light propagation direction. Our observed initial momentum offsets are well reproduced by the strong-field approximation (SFA). A mechanistic picture is introduced that links the measured non-adiabatic offset to the magnetic quantum number of virtually populated intermediate states.