Ultra Steep Spectrum (USS) radio sources have been successfully used to select powerful radio sources at high redshifts (z>~2). Typically restricted to large-sky surveys and relatively bright radio flux densities, it has gradually become possible to extend the USS search to sub-mJy levels, thanks to the recent appearance of sensitive low-frequency radio facilities. Here a first detailed analysis of the nature of the faintest USS sources is presented. By using Giant Metrewave Radio Telescope and Very Large Array radio observations of the Lockman Hole at 610 MHz and 1.4 GHz, a sample of 58 USS sources, with 610 MHz integrated fluxes above 100 microJy, is assembled. Deep infrared data at 3.6 and 4.5 micron from the Spitzer Extragalactic Representative Volume Survey (SERVS) is used to reliably identify counterparts for 48 (83%) of these sources, showing an average total magnitude of [3.6](AB)=19.8 mag. Spectroscopic redshifts for 14 USS sources, together with photometric redshift estimates, improved by the use of the deep SERVS data, for a further 19 objects, show redshifts ranging from z=0.1 to z=2.8, peaking at z~0.6 and tailing off at high redshifts. The remaining 25 USS sources, with no redshift estimate, include the faintest [3.6] magnitudes, with 10 sources undetected at 3.6 and 4.5 micron (typically [3.6]>22-23 mag, from local measurements), which suggests the likely existence of higher redshifts among the sub-mJy USS population. The comparison with the Square Kilometre Array Design Studies Simulated Skies models indicate that Fanaroff-Riley type I radio sources and radio-quiet Active Galactic Nuclei may constitute the bulk of the faintest USS population, and raises the possibility that the high efficiency of the USS technique for the selection of high redshift sources remains even at the sub-mJy level.