We study the pi N --> phi N reaction close to the phi N threshold within the chiral unitary approach, by combining the pi^- p --> K^+ Sigma^-, pi^- p --> K^0 Sigma^0 and pi^- p --> K^0 Lambda amplitudes with the coupling of the phi to the K components of the final states of these reactions via quantum loops. We obtain a good agreement with experiment when the dominant pi^- p --> K^0 Lambda amplitude is constrained with its experimental cross section. We also evaluate the coupling of the N*(1535) to phi N and find a moderate coupling as a consequence of partial cancellation of the large KY components of the N*(1535). We also show that the N*(1535) pole approximation is too small to reproduce the measured cross section for the pi N --> phi N reaction.