An important problem in analysis of neural data is to characterize interactions across brain regions from high-dimensional multiple-electrode recordings during a behavioral experiment. Lead-lag effects indicate possible directional flows of neural information, but they are often transient, appearing during short intervals of time. Such non-stationary interactions can be difficult to identify, but they can be found by taking advantage of the replication structure inherent to many neurophysiological experiments. To describe non-stationary interactions between replicated pairs of high-dimensional time series, we developed a method of estimating latent, non-stationary cross-correlation. Our approach begins with an extension of probabilistic CCA to the time series setting, which provides a model-based interpretation of multiset CCA. Because the covariance matrix describing non-stationary dependence is high-dimensional, we assume sparsity of cross-correlations within a range of possible interesting lead-lag effects. We show that the method can perform well in realistic settings and we apply it to 192 simultaneous local field potential (LFP) recordings from prefrontal cortex (PFC) and visual cortex (area V4) during a visual memory task. We find lead-lag relationships that are highly plausible, being consistent with related results in the literature.