We use K-band spectroscopic data from the Multi-Object Spectroscopic Emission Line (MOSEL) survey to analyze the kinematic properties of galaxies at z>3. Our sample consists of 34 galaxies at 3.0<zspec<3.8 between 9.0<log(M_star)<11.0. We find that galaxies with log(M_star) > 10.2 at z > 3 have 56 +/- 21 km/s lower integrated velocity dispersion compared to galaxies at z ~ 2 of similar stellar mass. Massive galaxies at z > 3 have either a flat or declining star formation history (SFH), whereas similar stellar mass galaxies at z~2.0 exhibit a slight peak in the past 500 Myrs. Comparing with the IllustrisTNG cosmological simulation, we find that (i) the dynamical mass of massive galaxies in simulations (log(M_star) > 10.0) increases by ~0.1 dex at a fixed stellar mass between z=2.0-3.0, and (ii) dynamical mass growth is coupled with a rapid rise in the ex-situ stellar mass fraction (stars accreted from other galaxies) for massive galaxies at z < 3.5. We speculate that the rising contribution of ex-situ stellar mass to the total stellar mass growth of massive galaxies is driving the higher integrated velocity dispersion and rising SFHs of massive galaxies at z~2.0 compared to galaxies of similar stellar masses at z > 3.