We revisit ground states of spinor Bose-Einstein condensates with a Rashba spin-orbit coupling, and find that votices show up as a direct consequence of spontaneous symmetry breaking into a combined gauge, spin, and space rotation symmetry, which determines the vortex-core spin state at the rotating center. For the continuous combined symmetry, the total spin rotation about the rotating axis is restricted to $2pi$, whereas for the discrete combined symmetry, we further need 2F quantum numbers to characterize the total spin rotation for the spin-$F$ system. For lattice phases we find that in the ground state the topological charge for each unit cell vanishes. However, we find two types of highly symmetric lattices with a nontrivial topological charge in the spin-$frac{1}{2}$ system based on the symmetry classification, and show that they are skyrmion crystals.