We present deep, continuum images of eleven high-redshift (0.811 < z < 1.875) 3CR radio galaxies observed with NICMOS. Our images probe the rest-frame optical light where stars are expected to dominate the galaxy luminosity. The rest-frame UV light of eight of these galaxies demonstrates the well-known ``alignment effect. Most of the radio galaxies have rounder, more symmetric morphologies at rest-frame optical wavelengths. Here we show the most direct evidence that in most cases the stellar hosts are normal elliptical galaxies with de Vaucouleurs law light profiles. For a few galaxies very faint traces of the UV-bright aligned component are also visible in the infrared images. We derive both the effective radius and surface-brightness for nine of eleven sample galaxies by fitting surface-brightness models to them. We find their sizes are similar to those of local FRII radio source hosts and are in general larger than other local galaxies. The derived host galaxy luminosities are very high and lie at the bright end of luminosity functions constructed at similar redshifts. The galaxies in our sample are also brighter than the rest-frame size--surface-brightness locus defined by the low-redshift sources. Passive evolution roughly aligns the z ~ 1 galaxies with the low-redshift samples. The optical host is sometimes centered on a local minimum in the rest-frame UV emission, suggesting the presence of substantial dust obscuration. We also see good evidence of nuclear point sources in three galaxies. Overall, our results are consistent with the hypothesis that these galaxies have already formed the bulk of their stars at redshifts greater than z >~ 2, and that the AGN phenomenon takes place within otherwise normal, perhaps passively evolving, galaxies. (abridged)