We present observations and dynamical models of the stellar nuclear clusters (NCs) at the centres of NGC 4244 and M33. We then compare these to an extensive set of simulations testing the importance of purely stellar dynamical mergers on the formation and growth of NCs. Mergers of star clusters are able to produce a wide variety of observed properties, including densities, structural scaling relations, shapes (including the presence of young discs) and even rapid rotation. Nonetheless, difficulties remain, most notably that the second order kinematic moment V_rms = (V^2 + sigma^2)^(1/2) of the models is too centrally peaked to match observations. This can be remedied by the merger of star clusters onto a pre-existing nuclear disc, but the line-of-sight velocity V is still more slowly rising than in NGC 4244. Our results therefore suggest that purely stellar dynamical mergers cannot form NCs, and that gas dissipation is a necessary ingredient for at least ~50% of a NCs mass. The negative vertical anisotropy found in NGC 4244 however requires at least 10% of the mass to be accreted as stars, since gas dissipation and in situ star formation leads to positive vertical anisotropy.