Ultra-steep spectrum (USS) radio sources are good tracers of powerful radio galaxies at $z > 2$. Identification of even a single bright radio galaxy at $z > 6$ can be used to detect redshifted 21cm absorption due to neutral hydrogen in the intervening IGM. Here we describe a new sample of high-redshift radio galaxy (HzRG) candidates constructed from the TGSS ADR1 survey at 150 MHz. We employ USS selection ($alpha le -1.3$) in $sim10000$ square degrees, in combination with strict size selection and non-detections in all-sky optical and infrared surveys. We apply flux density cuts that probe a unique parameter space in flux density ($50 < S_{textrm{150}} < 200$ mJy) to build a sample of 32 HzRG candidates. Follow-up Karl G. Jansky Very Large Array (VLA) observations at 1.4 GHz with an average beam size of $1.3$ arcseconds ($$) revealed $sim 48%$ of sources to have a single radio component. P-band (370 MHz) imaging of 17 of these sources revealed a flattening radio SED for ten sources at low frequencies, which is expected from compact HzRGs. Two of our sources lie in fields where deeper multi-wavelength photometry and ancillary radio data are available and for one of these we find a best-fit photo-z of $4.8 pm 2.0$. The other source has $z_{textrm{phot}}=1.4 pm 0.1$ and a small angular size ($3.7$), which could be associated with an obscured star forming galaxy or with a dead elliptical. One USS radio source not part of the HzRG sample but observed with the VLA nonetheless is revealed to be a candidate giant radio galaxy with a host galaxy photo-z of $1.8pm0.5$, indicating a size of 875 kpc.