Energy-time entangled photons are critical in many quantum optical phenomena and have emerged as important elements in quantum information protocols. Entanglement in this degree of freedom often manifests itself on ultrafast timescales making it very difficult to detect, whether one employs direct or interferometric techniques, as photon-counting detectors have insufficient time resolution. Here, we implement ultrafast photon counters based on nonlinear interactions and strong femtosecond laser pulses to probe energy-time entanglement in this important regime. Using this technique and single-photon spectrometers, we characterize all the spectral and temporal correlations of two entangled photons with femtosecond resolution. This enables the witnessing of energy-time entanglement using uncertainty relations and the direct observation of nonlocal dispersion cancellation on ultrafast timescales. These techniques are essential to understand and control the energy-time degree of freedom of light for ultrafast quantum optics.