Within the central 10pc of our Galaxy lies a dense nuclear star cluster (NSC), and similar NSCs are found in most nearby galaxies. Studying the structure and kinematics of NSCs reveals the history of mass accretion of galaxy nuclei. Because the Milky Way (MW) NSC is at a distance of only 8kpc, we can spatially resolve the MWNSC on sub-pc scales. This makes the MWNSC a reference object for understanding the formation of all NSCs. We have used the NIR long-slit spectrograph ISAAC (VLT) in a drift-scan to construct an integral-field spectroscopic map of the central 9.5 x 8pc of our Galaxy. We use this data set to extract stellar kinematics both of individual stars and from the unresolved integrated light spectrum. We present a velocity and dispersion map from the integrated light and model these kinematics using kinemetry and axisymmetric Jeans models. We also measure CO bandhead strengths of 1,375 spectra from individual stars. We find kinematic complexity in the NSCs radial velocity map including a misalignment of the kinematic position angle by 9 degree counterclockwise relative to the Galactic plane, and indications for a rotating substructure perpendicular to the Galactic plane at a radius of 20 or 0.8pc. We determine the mass of the NSC within r = 4.2pc to 1.4 x 10^7 Msun. We also show that our kinematic data results in a significant underestimation of the supermassive black hole (SMBH) mass. The kinematic substructure and position angle misalignment may hint at distinct accretion events. This indicates that the MWNSC grew at least partly by the mergers of massive star clusters. Compared to other NSCs, the MWNSC is on the compact side of the r_eff - M_NSC relation. The underestimation of the SMBH mass might be caused by the kinematic misalignment and a stellar population gradient. But it is also possible that there is a bias in SMBH mass measurements obtained with integrated light.