We present J-H-K photometry for a sample of 45 high redshift quasars found by the Sloan Digital Sky Survey. The sample was originally selected on the basis of optical colors and spans a redshift range from 3.6 to 5.03. Our photometry reflects the rest-frame SED longward of Ly alpha for all redshifts. The results show that the near-IR colors of high redshift quasars are quite uniform. We have modelled the continuum shape of the quasars (from just beyond Ly alpha to ~4000 A) with a power law of the form f_nu propto nu^alpha, and find <alpha > =-0.57 with a scatter of 0.33. This value is similar to what is found for lower redshift quasars over the same restframe wavelength range, and we conclude that there is hardly any evolution in the continuum properties of optically selected quasars up to redshift 5. The spectral indices found by combining near-IR with optical photometry are in general consistent but slightly flatter than what is found for the same quasars using the optical spectra and photometry alone, showing that the continuum region used to determine the spectral indices can somewhat influence the results.