The Galactic Ridge X-ray Emission (GRXE) is apparently extended X-ray emission along the Galactic Plane. The X-ray spectrum is characterized by hard continuum with a strong Fe K emission feature in the 6-7 keV band. A substantial fraction (~80%) of the GRXE in the Fe band was resolved into point sources by deep Chandra imaging observations, thus GRXE is mostly composed of dim Galactic X-ray point sources at least in this energy band. To investigate the populations of these dim X-ray point sources, we carried out Near-Infrared (NIR) follow-up spectroscopic observations in two deep Chandra fields located in the Galactic plane at (l,b)=(0.1{arcdeg}, -1.4{arcdeg}) and (28.5{arcdeg}, 0.0{arcdeg}) using NTT/SofI and Subaru/MOIRCS. We obtained well-exposed NIR spectra from 65 objects and found that there are three main classes of Galactic sources based on the X-ray color and NIR spectral features: those having (A) hard X-ray spectra and NIR emission features such as HI(Br{gamma}), HeI, and HeII (2 objects), (B) soft X-ray spectra and NIR absorption features such as HI, NaI, CaI, and CO (46 objects), and (C) hard X-ray spectra and NIR absorption features such as HI, NaI, CaI and CO (17 objects). From these features, we argue that class A sources are Cataclysmic Variables (CVs), and class B sources are late-type stars with enhanced coronal activity, which is in agreement with current knowledge. Class C sources possibly belong to a new group of objects, which has been poorly studied so far. We argue that the candidate sources for class C are the binary systems hosting white dwarfs and late-type companions with very low accretion rates. It is likely that this newly recognized class of the sources contribute to a non-negligible fraction of the GRXE, especially in the Fe K band.