Direct measurements on the temporal envelope of quantum light are a challenging task and not many examples are known since most classical pulse characterisation methods do not work on the single photon level. Knowledge of both spectrum and timing can however give insights on properties that cannot be determined by the spectrum alone. While temporal measurements on single photons on timescales of tens of picoseconds are possible with superconducting photon detectors and picosecond measurements have been performed using streak cameras, there are no commercial single photon sensitive devices with femtosecond resolution available. While time-domain sampling using sum-frequency generation has been already exploited for such measurement, inefficient conversion has necessitated long integration times to build the temporal profile. We demonstrate a highly efficient waveguided sum-frequency generation process in Lithium Niobate to measure the temporal envelope of single photons with femtosecond resolution with short enough acquisition time to provide a live-view of the measurement. We demonstrate the measurement technique and combine it with spectral measurements using a dispersive fiber time-of-flight spectrometer to determine upper and lower bounds for the spectral purity of heralded single photons. The approach complements the joint spectral intensity measurements as a measure on the purity can be given without knowledge of the spectral phase.