Non-Markovian effects in the evolution of open quantum systems have recently attracted widespread interest, particularly in the context of assessing the efficiency of energy and charge transfer in nanoscale biomolecular networks and quantum technologies. With the aid of many-body simulation methods, we uncover and analyse an ultrafast environmental process that causes energy relaxation in the reduced system to depend explicitly on the phase relation of the initial state preparation. Remarkably, for particular phases and system parameters, the net energy flow is uphill, transiently violating the principle of detailed balance, and implying that energy is spontaneously taken up from the environment. A theoretical analysis reveals that non-secular contributions, significant only within the environmental correlation time, underlie this effect. This suggests that environmental energy harvesting will be observable across a wide range of coupled quantum systems.