We report the multi-wavelength identification of the X-ray sources found in the Subaru-XMM-Newton Deep Survey (SXDS) using deep imaging data covering the wavelength range between the far-UV to the mid-IR. We select a primary counterpart of each X-ray source by applying the likelihood ratio method to R-band, 3.6micron, near-UV, and 24micron source catalogs as well as matching catalogs of AGN candidates selected in 1.4GHz radio and i-band variability surveys. Once candidates of Galactic stars, ultra-luminous X-ray sources in a nearby galaxy, and clusters of galaxies are removed there are 896 AGN candidates in the sample. We conduct spectroscopic observations of the primary counterparts with multi-object spectrographs in the optical and NIR; 65% of the X-ray AGN candidates are spectroscopically-identified. For the remaining X-ray AGN candidates, we evaluate their photometric redshift with photometric data in 15 bands. Utilising the multi-wavelength photometric data of the large sample of X-ray selected AGNs, we evaluate the stellar masses, M*, of the host galaxies of the narrow-line AGNs. The distribution of the stellar mass is remarkably constant from z=0.1 to 4.0. The relation between M* and 2--10 keV luminosity can be explained with strong cosmological evolution of the relationship between the black hole mass and M*. We also evaluate the scatter of the UV-MIR spectral energy distribution (SED) of the X-ray AGNs as a function of X-ray luminosity and absorption to the nucleus. The scatter is compared with galaxies which have redshift and stellar mass distribution matched with the X-ray AGN. The UV-NIR SEDs of obscured X-ray AGNs are similar to those of the galaxies in the matched sample. In the NIR-MIR range, the median SEDs of X-ray AGNs are redder, but the scatter of the SEDs of the X-ray AGN broadly overlaps that of the galaxies in the matched sample.