In the path integral formulation of the evolution of an open quantum system coupled to a Gaussian, non-interacting environment, the dynamical contribution of the latter is encoded in an object called the influence functional. Here, we relate the influence functional to the process tensor -- a more general representation of a quantum stochastic process -- describing the evolution. We then use this connection to motivate a tensor network algorithm for the simulation of multi-time correlations in open systems, building on recent work where the influence functional is represented in terms of time evolving matrix product operators. By exploiting the symmetries of the influence functional, we are able to use our algorithm to achieve orders-of-magnitude improvement in the efficiency of the resulting numerical simulation. Our improved algorithm is then applied to compute exact phonon emission spectra for the spin-boson model with strong coupling, demonstrating a significant divergence from spectra derived under commonly used assumptions of memorylessness.