We use the multifractal formalism to describe the effects of dissipation on Lagrangian velocity statistics in turbulent flows. We analyze high Reynolds number experiments and direct numerical simulation (DNS) data. We show that this approach reproduces the shape evolution of velocity increment probability density functions (PDF) from Gaussian to stretched exponentials as the time lag decreases from integral to dissipative time scales. A quantitative understanding of the departure from scaling exhibited by the magnitude cumulants, early in the inertial range, is obtained with a free parameter function D(h) which plays the role of the singularity spectrum in the asymptotic limit of infinite Reynolds number. We observe that numerical and experimental data are accurately described by a unique quadratic D(h) spectrum which is found to extend from $h_{min} approx 0.18$ to $h_{max} approx 1$, as the signature of the highly intermittent nature of Lagrangian velocity fluctuations.