We point out that the remarkable linearity of the ultra-steep radio spectra of high redshift radio galaxies reflects a previously reported general trend for powerful radio galaxies, according to which the spectral curvature is lesser for sources having steeper spectra (measured near rest-frame 1 GHz). We argue based on existing theoretical and observational evidence that it is premature to conclude that the particle acceleration mechanism in sources having straight, ultra-steep radio spectra gives rise to an ultra-steep injection spectrum of the radiating electrons. In empirical support to this we show that the estimated injection spectral indices, available for a representative sample of 35 compact steep spectrum (CSS) radio sources are not correlated with their rest-frame (intrinsic) rotation measures, which are known to be typically large, indicating a dense environment, as is also the case for high-$z$ radio galaxies.