We review the most recent developments in the theory of open quantum systems focusing on situations in which the reservoir memory effects, due to long-lasting and non-negligible correlations between system and environment, play a crucial role. These systems are often referred to as non-Markovian systems. After a brief summary of different measures of non-Markovianity that have been introduced over the last few years we restrict our analysis to the investigation of information flow between system and environment. Within this framework we introduce an important application of non-Markovianity, namely its use as a quantum probe of complex quantum systems. To illustrate this point we consider quantum probes of ultracold gases, spin chains, and trapped ion crystals and show how properties of these systems can be extracted by means of non-Markovianity measures.