Ultra Steep Spectrum (USS) radio sources are one of the efficient tracers of powerful High-z Radio Galaxies (HzRGs). In contrast to searches for powerful HzRGs from radio surveys of moderate depths, fainter USS samples derived from deeper radio surveys can be useful in finding HzRGs at even higher redshifts and in unveiling a population of obscured weaker radio-loud AGN at moderate redshifts. Using our 325 MHz GMRT observations (5-sigma ~ 800 microJy) and 1.4 GHz VLA observations (5-sigma ~ 80 - 100 microJy) available in two subfields (viz., VLA-VIMOS VLT Deep Survey (VLA-VVDS) and Subaru X-ray Deep Field (SXDF)) of the XMM-LSS field, we derive a large sample of 160 faint USS radio sources and characterize their nature. The optical, IR counterparts of our USS sample sources are searched using existing deep surveys, at respective wavelengths. We attempt to unveil the nature of our faint USS sources using diagnostic techniques based on mid-IR colors, flux ratios of radio to mid-IR, and radio luminosities. Redshift estimates are available for 86/116 (~ 74%) USS sources in the VLA-VVDS field and for 39/44 (~ 87%) USS sources in the SXDF fields with median values (z_median) ~ 1.18 and ~ 1.57, which are higher than that for non-USS radio sources (z_median non-USS ~ 0.99 and ~ 0.96), in the two subfields, respectively. The flux ratio of radio to mid-IR (S_1.4 GHz/S_3.6 micron) versus redshift diagnostic plot suggests that more than half of our USS sample sources distributed over z ~ 0.5 to 3.8 are likely to be hosted in obscured environments. A significant fraction (~ 26% in the VLA-VVDS and ~ 13% in the SXDF) of our USS sources without redshift estimates mostly remain unidentified in the existing optical, IR surveys, and exhibit high radio to mid-IR flux ratio limits similar to HzRGs, and thus, can be considered as potential HzRG candidates.