We investigate the upper chromosphere and the transition region of the sunspot umbra using the radio brightness temperature at 34 GHz (corresponding to 8.8-mm observations) as observed by the Nobeyama Radioheliograph (NoRH). Radio free-free emission in the longer millimeter range is generated around the transition region, and its brightness temperature yields the regions temperature and density distribution. We use the NoRH data at 34 GHz by applying the Steer-CLEAN image synthesis. These data and the analysis method enable us to investigate the chromospheric structures in the longer millimeter range with high spatial resolution and sufficient visibilities. We also perform simultaneous observations of one sunspot using the NoRH and the Nobeyama 45-m telescope operating at 115 GHz. We determine that 115-GHz emission mainly originates from the lower chromosphere while 34-GHz emission mainly originates from the upper chromosphere and transition region. These observational results are consistent with the radio emission characteristics estimated from the current atmospheric models of the chromosphere. On the other hand, the observed brightness temperature of the umbral region is almost the same as that of the quiet region. This result is inconsistent with the current sunspot models, which predict a considerably higher brightness temperature of the sunspot umbra at 34 GHz. This inconsistency suggests that the temperature of the region at which the 34 GHz radio emission becomes optically thick should be lower than that predicted by the models.