The X-shooter Spectral Library (XSL) is an empirical stellar library at medium spectral resolution covering the wavelength range from 3000 AA to 24 800 AA. This library aims to provide a benchmark for stellar population studies. In this work, we present a uniform set of stellar atmospheric parameters, effective temperatures, surface gravities, and iron abundances for 754 spectra of 616 XSL stars. We used the full-spectrum fitting package ULySS with the empirical MILES library as reference to fit the ultraviolet-blue (UVB) and visible (VIS) spectra. We tested the internal consistency and we compared our results with compilations from the literature. The stars cover a range of effective temperature 2900 < Teff < 38 000 K, surface gravity 0 < log g < 5.7, and iron abundance -2.5 < [Fe/H] < +1.0, with a couple of stars extending down to [Fe/H] = -3.9. The precisions of the measurements for the G- and K-type stars are 0.9%, 0.14, and 0.06 in Teff, log g, and [Fe/H], respectively. For the cool giants with log g < 1, the precisions are 2.1%, 0.21, and 0.22, and for the other cool stars these values are 1%, 0.14, and 0.10. For the hotter stars (Teff > 6500 K), these values are 2.6%, 0.20, and 0.10 for the three parameters.