We consider the problem of extrapolating the perturbation series for the dilute Fermi gas in three dimensions to the unitary limit of infinite scattering length and into the BEC region, using the available strong-coupling information to constrain the extrapolation problem. In this constrained extrapolation problem (CEP) the goal is to find classes of approximants that give well converged results already for low perturbative truncation orders. First, we show that standard Pad{e} and Borel methods are too restrictive to give satisfactory results for this CEP. A generalization of Borel extrapolation is given by the so-called Maximum Entropy extrapolation method (MaxEnt). However, we show that MaxEnt requires extensive elaborations to be applicable to the dilute Fermi gas and is thus not practical for the CEP in this case. Instead, we propose order-dependent-mapping extrapolation (ODME) as a simple, practical, and general method for the CEP. We find that the ODME approximants for the ground-state energy of the dilute Fermi gas are robust with respect to changes of the mapping choice and agree with results from quantum Monte Carlo simulations within uncertainties.