Graphene in the quantum Hall regime exhibits a multi-component structure due to the electronic spin and chirality degrees of freedom. While the applied field breaks the spin symmetry explicitly, we show that the fate of the chirality SU(2) symmetry is more involved: the leading symmetry-breaking terms differ in origin when the Hamiltonian is projected onto the central (n=0) rather than any of the other Landau levels. Our description at the lattice level leads to a Harper equation; in its continuum limit, the ratio of lattice constant a and magnetic length l_B assumes the role of a small control parameter in different guises. The leading symmetry-breaking terms are direct (n=0) and exchange (n different from 0) terms, which are algebraically small in a/l_B. We comment on the Haldane pseudopotentials for graphene, and evaluate the easy-plane anisotropy of the graphene ferromagnet.