We calculate the charge quadrupole and magnetic octupole moments of baryons using a group theoretical approach based on broken SU(6) spin-flavor symmetry. The latter is an approximate symmetry of the QCD Lagrangian which becomes exact in the large color N_c limit. Spin-flavor symmetry breaking is induced by one-, two-, and three-quark terms in the electromagnetic current operator. Two- and three-quark currents provide the leading contributions for higher multipole moments, despite being of higher order in an 1/N_c expansion. Our formalism leads to relations between N --> N* transition multipole moments and nucleon ground state properties. We compare our results to experimental quadrupole and octupole transition moments extracted from measured helicity amplitudes.