Penetrating probes in heavy-ion collisions, like jets and photons, are sensitive to the transport coefficients of the produced quark-gluon plasma, such as shear and bulk viscosity. Quantifying this sensitivity requires a detailed understanding of photon emission and jet-medium interaction in a non-equilibrium plasma. Up to now, such an understanding has been hindered by plasma instabilities which arise out of equilibrium and lead to spurious divergences when evaluating the rate of interaction of hard probes with the plasma. In this paper, we show that taking into account the time evolution of an unstable plasma cures these divergences. We calculate the time evolution of gluon two-point correlators in a setup with small initial momentum anisotropy and show that the gluon occupation density grows exponentially at early times. Based on this calculation, we argue for a phenomenological prescription where instability poles are subtracted. Finally, we show that in the Abelian case instability fields do not affect medium-induced photon emission to our order of approximation.