We introduce a new method for performing ultrafast imaging and tracking of surface plasmon wave packets that propagate on metal films. We demonstrate the efficiency of leakage radiation microscopy implemented in the time domain for measuring both group and phase velocities of near-field pulses with a high level of precision. The versatility of our far-field imaging method is particularly appealing in the context of ultrafast near-field optics.