Galaxies above redshift 1 can be very clumpy, with irregular morphologies dominated by star complexes as large as 2 kpc and as massive as a few 10^8 or 10^9 Mo. Their co-moving densities and rapid evolution suggest that most present-day spirals could have formed through a clumpy phase. The clumps may form by gravitational instabilities in gas-rich turbulent disks; they do not appear to be separate galaxies merging together. We show here that the formation of the observed clumps requires initial disks of gas and stars with almost no stabilizing bulge or stellar halo. This cannot be achieved in models where disk galaxies grow by mergers. Mergers tend to make stellar spheroids even when the gas fraction is high, and then the disk is too stable to make giant clumps. The morphology of high-redshift galaxies thus suggests that inner disks assemble mostly by smooth gas accretion, either from cosmological flows or from the outer disk during a grazing interaction.