Transient near-fields around metallic nanotips drive many applications, including the generation of ultrafast electron pulses and their use in electron microscopy. We have investigated the electron emission from a gold nanotip driven by mid-infrared few-cycle laser pulses. We identify a low-energy peak in the kinetic energy spectrum and study its shift to higher energies with increasing laser intensities from $1.7$ to $3.7cdot10^{11} mathrm{W}/mathrm{cm}^2$. The experimental observation of the upshift of the low-energy peak is compared to a simple model and numerical simulations, which show that the decay of the near-field on a nanometer scale results in non-adiabatic transfer of the ponderomotive potential to the kinetic energy of emitted electrons and in turn to a shift of the peak. We derive an analytic expression for the non-adiabatic ponderomotive shift, which, after the previously found quenching of the quiver motion, completes the understanding of the role of inhomogeneous fields in strong-field photoemission from nanostructures.