We are building a new spectral library with the X-Shooter instrument on ESOs VLT: XSL, the X-Shooter Spectral Library. We present our progress in building XSL, which covers the wavelength range from the near-UV to the near-IR with a resolution of R ~ 10000. At the time of writing we have collected spectra for nearly 240 stars. An important feature of XSL is that we have already collected spectra of more than 100 Asymptotic Giant Branch stars in the Galaxy and the Magellanic Clouds.