We present an analysis of X-ray and ultra-violet data of the dwarf nova VW Hyi that were obtained with XMM-Newton during the quiescent state. The X-ray spectrum indicates the presence of an optically thin plasma in the boundary layer that cools as it settles onto the white dwarf. The plasma has a continuous temperature distribution that is well described by a power-law or a cooling flow model with a maximum temperature of 6-8 keV. We estimate from the X-ray spectrum a boundary layer luminosity of 8*10^30 erg/s, which is only 20 per cent of the disk luminosity. The rate of accretion onto the white dwarf is 5*10^-12 solar masses per year, about half of the rate in the disk. From the high-resolution X-ray spectra, we estimate that the X-ray emitting part of the boundary layer is rotating with a velocity of 540 km/s, which is close the rotation velocity of the white dwarf but significantly smaller than the Keplerian velocity. We detect a 60-s quasi-periodic oscillation of the X-ray flux that is likely due to the rotation of the boundary layer. The X-ray and the ultra-violet flux show strong variability on a time scale of ~1500 s. We find that the variability in the two bands is correlated and that the X-ray fluctuations are delayed by ~100 s. The correlation indicates that the variable ultra-violet flux is emitted near the transition region between the disk and the boundary layer and that accretion rate fluctuations in this region are propagated to the X-ray emitting part of the boundary layer within ~100 s. An orbital modulation of the X-ray flux suggests that the inner accretion disk is tilted with respect to the orbital plane. The elemental abundances in the boundary layer are close to their solar values.