Pulsar timing array experiments search for phenomena that produce angular correlations in the arrival times of signals from millisecond pulsars. The primary goal is to detect an isotropic and stochastic gravitational wave background. We use simulated data to show that this search can be affected by the presence of other spatially correlated noise, such as errors in the reference time standard, errors in the planetary ephemeris, the solar wind and instrumentation issues. All these effects can induce significant false detections of gravitational waves. We test mitigation routines to account for clock errors, ephemeris errors and the solar wind. We demonstrate that it is non-trivial to find an effective mitigation routine for the planetary ephemeris and emphasise that other spatially correlated signals may be present in the data.