We present spectroscopic and eleven-band photometric redshifts for galaxies in the 100-uJy Subaru/XMM-Newton Deep Field radio source sample. We find good agreement between our redshift distribution and that predicted by the SKA Simulated Skies project. We find no correlation between K-band magnitude and radio flux, but show that sources with 1.4-GHz flux densities below ~1mJy are fainter in the near-infrared than brighter radio sources at the same redshift, and we discuss the implications of this result for spectroscopically-incomplete samples where the K-z relation has been used to estimate redshifts. We use the infrared--radio correlation to separate our sample into radio-loud and radio-quiet objects and show that only radio-loud hosts have spectral energy distributions consistent with predominantly old stellar populations, although the fraction of objects displaying such properties is a decreasing function of radio luminosity. We calculate the 1.4-GHz radio luminosity function (RLF) in redshift bins to z=4 and find that the space density of radio sources increases with lookback time to z~2, with a more rapid increase for more powerful sources. We demonstrate that radio-loud and radio-quiet sources of the same radio luminosity evolve very differently. Radio-quiet sources display strong evolution to z~2 while radio-loud AGNs below the break in the radio luminosity function evolve more modestly and show hints of a decline in their space density at z>1, with this decline occurring later for lower-luminosity objects. If the radio luminosities of these sources are a function of their black hole spins then slowly-rotating black holes must have a plentiful fuel supply for longer, perhaps because they have yet to encounter the major merger that will spin them up and use the remaining gas in a major burst of star formation.