We analyze the non-Markovian character of the dynamics of an open two-level atom interacting with a gas of ultra-cold fermions. In particular, we discuss the connection between the phenomena of orthogonality catastrophe and Fermi edge singularity occurring in such a kind of environment and the memory-keeping effects which are displayed in the time evolution of the open system.