In graphene growth, island symmetry can become lower than the intrinsic symmetries of both graphene and the substrate. First-principles calculations and Monte Carlo modeling explain the shapes observed in our experiments and earlier studies for various metal surface symmetries. For equilibrium shape, edge energy variations $delta E$ manifest in distorted hexagons with different ground-state edge structures. In growth or nucleation, energy variation enters exponentially as $sim e^{delta E / k_{B} T}$, strongly amplifying the symmetry breaking, up to completely changing the shapes to triangular, ribbon-like, or rhombic.