Pristine, atomically-cooled haloes are leading contenders for the sites of primordial quasar formation because atomic cooling triggers rapid baryon collapse that can create 10$^4$ - 10$^5$ M$_{odot}$ black hole seeds. However, until now no numerical simulations with a wide range of halo spins and assembly histories have followed the collapse for the times required to form a black hole. We have now performed cosmological simulations of baryon collapse in atomically-cooled haloes for times that are sufficient for supermassive stars to form and die as direct-collapse black holes (DCBHs). Our simulations reveal that fragmentation of the accretion disk at the center of the halo after $sim$ 500 kyr is nearly ubiquitous and in most cases leads to the formation of binary or multiple supermassive stellar systems. They also confirm that rapid baryon collapse proceeds for the times required for these stars to form DCBHs. Our discovery raises the exciting possibility of detecting gravitational waves from DCBH mergers with LISA and tidal disruption events in the near infrared with the James Webb Space Telescope and ground-based telescopes in the coming decade.