It is known that one can characterize the decoherence strength of a Markovian environment by the product of its temperature and induced damping, and order the decoherence strength of multiple environments by this quantity. We show that for non-Markovian environments in the weak coupling regime there also exists a natural (albeit partial) ordering of environment-induced irreversibility within a perturbative treatment. This measure can be applied to both low-temperature and non-equilibrium environments.