Quantum extremal islands reproduce the unitary Page curve of an evaporating black hole. This has been derived by including replica wormholes in the gravitational path integral, but for the transient, evaporating black holes most relevant to Hawkings paradox, these wormholes have not been analyzed in any detail. In this paper we study replica wormholes for black holes formed by gravitational collapse in Jackiw-Teitelboim gravity, and confirm that they lead to the island rule for the entropy. The main technical challenge is that replica wormholes rely on a Euclidean path integral, while the quantum extremal islands of an evaporating black hole exist only in Lorentzian signature. Furthermore, the Euclidean equations are non-local, so it is unclear how to bridge the gap between the Euclidean path integral and the local, Lorentzian dynamics of an evaporating black hole. We address these issues with Schwinger-Keldysh techniques and show how the non-local equations reduce to the local `boundary particle description in special cases.