We report on a statistical study of the 51 radio galaxies at the millijansky flux level from the Faint Images of the Radio Sky at Twenty centimeters, including their optical morphologies and structure obtained with the Hubble Space Telescope. Our optical imaging is significantly deeper (~2 mag) than previous studies with the superior angular resolution of space-based imaging. We that find 8/51 (16%) of the radio sources have no optically identifiable counterpart to AB~24 mag. For the remaining 43 sources, only 25 are sufficiently resolved in the HST images to reliably assign a visual classification: 15 (60%) are elliptical galaxies, 2 (8%) are late-type spiral galaxies, 1 (4%) is an S0, 3 (12%) are point-like objects (quasars), and 4 (16%) are merger systems. We find a similar distribution of optical types with measurements of the Sersic index. The optical magnitude distribution of these galaxies peaks at I~20.7+-0.5 AB mag, which is ~3 mag brighter than the depth of our typical HST field and is thus not due to the WFPC2 detection limit. This supports the luminosity-dependent density evolutionary model, where the majority of faint radio galaxies typically have L*-optical luminosities and a median redshift of z~0.8 with a relatively abrupt redshift cut-off at z>~2. We discuss our results in the context of the evolution of elliptical galaxies and active galactic nuclei.