We consider the origin of compact, short-period, Jupiter-mass planets. We propose that their diverse structure is caused by giant impacts of embryos and super-Earths or mergers with other gas giants during the formation and evolution of these hot Jupiters. Through a series of numerical simulations, we show that typical head-on collisions generally lead to total coalescence of impinging gas giants. Although extremely energetic collisions can disintegrate the envelope of gas giants, these events seldom occur. During oblique and moderately energetic collisions, the merger products retain higher fraction of the colliders cores than their envelopes. They can also deposit considerable amount of spin angular momentum to the gas giants and desynchronize their spins from their orbital mean motion. We find that the oblateness of gas giants can be used to infer the impact history. Subsequent dissipation of stellar tide inside the planets envelope can lead to runaway inflation and potentially a substantial loss of gas through Roche-lobe overflow. The impact of super-Earths on parabolic orbits can also enlarge gas giant planets envelope and elevates their tidal dissipation rate over $sim $ 100 Myr time scale. Since giant impacts occur stochastically with a range of impactor sizes and energies, their diverse outcomes may account for the dispersion in the mass-radius relationship of hot Jupiters.