New instrumental capabilities and the wealth of astrophysical information extractable from the near-infrared wavelength region have led to a growing interest in the field of high resolution spectroscopy at 1-5 mu. We aim to provide a library of observed high-resolution and high signal-to-noise-ratio near-infrared spectra of stars of various types throughout the Hertzsprung-Russell diagram. This is needed for the exploration of spectral features in this wavelength range and for comparison of reference targets with observations and models. High quality spectra were obtained using the CRIRES near-infrared spectrograph at ESOs VLT covering the range from 0.97 to 5.3 mu at high spectral resolution. Accurate wavelength calibration and correction for of telluric lines were performed by fitting synthetic transmission spectra for the Earths atmosphere to each spectrum individually. We describe the observational strategy and the current status and content of the library which includes 13 objects. The first examples of finally reduced spectra are presented. This publication will serve as a reference paper to introduce the library to the community and explore the extensive amount of material.