We present a catalogue and atlas of low-resolution spectra of a well defined sample of 341 objects in the FORS Deep Field. All spectra were obtained with the FORS instruments at the ESO VLT with essentially the same spectroscopic set-up. The observed extragalactic objects cover the redshift range 0.1 to 5.0. 98 objects are starburst galaxies and QSOs at z > 2. Using this data set we investigated the evolution of the characteristic spectral properties of bright starburst galaxies and their mutual relations as a function of the redshift. Significant evolutionary effects were found for redshifts 2 < z < 4. Most conspicuous are the increase of the average C IV absorption strength, of the dust reddening, and of the intrinsic UV luminosity, and the decrease of the average Ly alpha emission strength with decreasing redshift. In part the observed evolutionary effects can be attributed to an increase of the metallicity of the galaxies with cosmic age. Moreover, the increase of the total star-formation rates and the stronger obscuration of the starburst cores by dusty gas clouds suggest the occurrence of more massive starbursts at later cosmic epochs.