Zeeman deceleration is an experimental technique in which inhomogeneous, time-dependent magnetic fields generated inside an array of solenoid coils are used to manipulate the velocity of a supersonic beam. A 12-stage Zeeman decelerator has been built and characterized using hydrogen atoms as a test system. The instrument has several original features including the possibility to replace each deceleration coil individually. In this article, we give a detailed description of the experimental setup, and illustrate its performance. We demonstrate that the overall acceptance in a Zeeman decelerator can be significantly increased with only minor changes to the setup itself. This is achieved by applying a rather low, anti-parallel magnetic field in one of the solenoid coils that forms a temporally varying quadrupole field, and improves particle confinement in the transverse direction. The results are reproduced by three-dimensional numerical particle trajectory simulations thus allowing for a rigorous analysis of the experimental data. The findings suggest the use of a modified coil configuration to improve transverse focusing during the deceleration process.