The stellar initial mass function (IMF) plays a crucial role in determining the number of surviving stars in galaxies, the chemical composition of the interstellar medium, and the distribution of light in galaxies. A key unsolved question is whether the IMF is universal in time and space. Here we use state-of-the-art results of stellar evolution to show that the IMF of our Galaxy made a transition from an IMF dominated by massive stars to the present-day IMF at an early phase of the Galaxy formation. Updated results from stellar evolution in a wide range of metallicities have been implemented in a binary population synthesis code, and compared with the observations of carbon-enhanced metal-poor (CEMP) stars in our Galaxy. We find that applying the present-day IMF to Galactic halo stars causes serious contradictions with four observable quantities connected with the evolution of AGB stars. Furthermore, a comparison between our calculations and the observations of CEMP stars may help us to constrain the transition metallicity for the IMF which we tentatively set at [Fe/H] = -2. A novelty of the current study is the inclusion of mass loss suppression in intermediate-mass AGB stars at low-metallicity. This significantly reduces the overproduction of nitrogen-enhanced stars that was a major problem in using the high-mass star dominated IMF in previous studies. Our results also demonstrate that the use of the present day IMF for all time in chemical evolution models results in the overproduction of Type I.5 supernovae. More data on stellar abundances will help to understand how the IMF has changed and what caused such a transition.