We analyse the kinematic and chemical evolution of 203 distant spheroidal (elliptical and S0) galaxies at 0.2<z<0.8 which are located in different environments (rich clusters, low-mass clusters and in the field). VLT/FORS and CAHA/MOSCA spectra with intermediate-resolution have been acquired to measure the internal kinematics and stellar populations of the galaxies. From HST/ACS and WFPC2 imaging, surface brightness profiles and structural parameters were derived for half of the galaxy sample. The scaling relations of the Faber-Jackson relation and Kormendy relation as well as the Fundamental Plane indicate a moderate evolution for the whole galaxy population in each density regime. In all environments, S0 galaxies show a faster evolution than elliptical galaxies. For the cluster galaxies a slight radial dependence of the evolution out to one virial radius is found. Dividing the samples with respect to their mass, a mass dependent evolution with a stronger evolution of lower-mass galaxies (M<2x10^{11} M_{sun}) is detected. Evidence for recent star formation is provided by blue colours and weak OII emission or strong Hdelta absorption features in the spectra. The results are consistent with a down-sizing formation scenario which is independent from the environment of the galaxies.