We compare overlap fermions, which are chirally invariant, and Wilson twisted mass fermions in the approach to the chiral limit. Our quenched simulations reveal that with both formulations of lattice fermions pion masses of O(250 MeV) can be reached in practical simulations. Our comparison is done at a fixed lattice spacing a=0.123 fm. Several quantities are measured, such as hadron masses and pseudoscalar decay constants.