Many galaxies at high redshift have peculiar morphologies dominated by 10^8-10^9 Mo kpc-sized clumps. Using numerical simulations, we show that these clump clusters can result from fragmentation in gravitationally unstable primordial disks. They appear as chain galaxies when observed edge-on. In less than 1 Gyr, clump formation, migration, disruption, and interaction with the disk cause these systems to evolve from initially uniform disks into regular spiral galaxies with an exponential or double-exponential disk profile and a central bulge. The inner exponential is the initial disk size and the outer exponential is from material flung out by spiral arms and clump torques. A nuclear black hole may form at the same time as the bulge from smaller black holes that grow inside the dense cores of each clump. The properties and lifetimes of the clumps in our models are consistent with observations of the clumps in high redshift galaxies, and the stellar motions in our models are consistent with the observed velocity dispersions and lack of organized rotation in chain galaxies. We suggest that violently unstable disks are the first step in spiral galaxy formation. The associated starburst activity gives a short timescale for the initial stellar disk to form.