This paper reports that the X-ray spectrum from the Galactic Center X-ray Emission (GCXE) is expressed by the assembly of active binaries, non-magnetic Cataclysmic Variables, magnetic Cataclysmic Variables (X-ray active star: XAS), cold matter and diffuse sources. In the fitting of the limited components of the XASs, the GCXE spectrum exhibits significant excesses with $chi^2/d.o.f. =5.67$. The excesses are found at the energies of K$alpha$, He$alpha$, Ly$alpha$ and radiative recombination continuum of S, Fe and Ni. By adding components of the cold matter and the diffuse sources, the GCXE spectrum is nicely reproduced with $chi^2/d.o.f. = 1.53$, which is a first quantitative model for the origin of the GCXE spectrum. The drastic improvement is mainly due to the recombining plasmas in the diffuse sources, which indicate the presence of high-energy activity of Sgr A$^*$ in the past of $> 1000$~years.