We investigate numerically the time dependence of window overlaps in a three-dimensional Ising spin glass below its transition temperature after a rapid quench. Using an efficient GPU implementation, we are able to study large systems up to lateral length $L=128$ and up to long times of $t=10^8$ sweeps. We find that the data scales according to the ratio of the window size $W$ to the non-equilibrium coherence length $xi(t)$. We also show a substantial change in behavior if the system is run for long enough that it globally equilibrates, i.e. $xi(t) approx L/2$, where $L$ is the lattice size. This indicates that the local behavior of a spin glass depends on the spin configurations (and presumably also the bonds) far away. We compare with similar simulations for the Ising ferromagnet. Based on these results, we speculate on a connection between the non-equilibrium dynamics discussed here and averages computed theoretically using the metastate.