Several recent studies of energy transfer in photosynthetic light harvesting complexes have revealed a subtle interplay between coherent and decoherent dynamic contributions to the overall transfer efficiency in these open quantum systems. In this work we systematically investigate the impact of temporal and spatial correlations in environmental fluctuations on excitation transport in the Fenna-Matthews-Olson photosynthetic complex. We demonstrate that the exact nature of the correlations can have a large impact on the efficiency of light harvesting. In particular, we find that (i) spatial correlations can enhance coherences in the site basis while at the same time slowing transport, and (ii) the overall efficiency of transport is optimized at a finite temporal correlation that produces maximum overlap between the environmental power spectrum and the excitonic energy differences, which in turn results in enhanced driving of transitions between excitonic states.