In this letter, we show that the non-linearitites of large amplitude motions in atomic nuclei induce giant quadrupole and monopole vibrations. As a consequence, the main source of anharmonicity is the coupling with configurations including one of these two giant resonances on top of any state. Two-phonon energies are often lowered by one or two MeV because of the large matrix elements with such three phonon configurations. These effects are studied in two nuclei, 40Ca and 208Pb.