The electric-field control of $d$-electron magnetism in multiferroic transition metal oxides is attracting widespread interest for the underlying fundamental physics and for next generation spintronic devices. Here, we report an extensive study of the $3d$ magnetism in magnetoelectric Ga$_{0.6}$Fe$_{1.4}$O$_3$ (GFO) epitaxial films by polarization dependent x-ray absorption spectroscopy. We found a non-zero integral of the x-ray magnetic circular dichroism, with a sign depending upon the relative orientation between the external magnetic field and the crystallographic axes. %By reliably enlarging the limit of the spin and orbital sum rules, which usually holds for materials where the magnetic ions exhibit a unique crystal field symmetry This finding translates in a sign-reversal between the average Fe magnetic orbital and spin moments. Large Fe-displacements, among some of the octahedral sites, lower the symmetry of the system producing anisotropic paths for the Fe-O bondings giving rise to a large orbital-lattice interaction akin to a preferential crystallographic direction for the magnetic orbital moment. The latter may lead to a partial re-orientation of the magnetic orbital moment under an external magnetic field that, combined to the ferrimagnetic nature of the GFO, can qualitatively explain the observed sign-reversal of the XMCD integral. The results suggest that a control over the local symmetry of the oxygen octahedra in transition metal oxides can offer a suitable leverage over the manipulation of the effective orbital and spin moments in magnetoelectric systems.