In this review, we discuss the properties of a few impurity atoms immersed in a gas of ultracold fermions, the so-called Fermi polaron problem. On one side, this many-body system is appealing because it can be described almost exactly with simple diagrammatic and/or variational theoretical approaches. On the other, it provides quantitatively reliable insight into the phase diagram of strongly interacting population imbalanced quantum mixtures. In particular, we show that the polaron problem can be applied to study itinerant ferromagnetism, a long standing problem in quantum mechanics.