The generation of ultrafast laser pulses and the reconstruction of their electric fields is essential for many applications in modern optics. Quantum optical fields can also be generated on ultrafast time scales, however, the tools and methods available for strong laser pulses are not appropriate for measuring the properties of weak, possibly entangled pulses. Here, we demonstrate a method to reconstruct the joint-spectral amplitude of a two-photon energy-time entangled state from joint measurements of the frequencies and arrival times of the photons, and the correlations between them. Our reconstruction method is based on a modified Gerchberg-Saxton algorithm. Such techniques are essential to measure and control the shape of ultrafast entangled photon pulses.