Semiconductor nano-devices have been scaled to the level that transport can be dominated by a single dopant atom. In the strong coupling case a Kondo effect is observed when one electron is bound to the atom. Here, we report on the spin as well as orbital Kondo ground state. We experimentally as well than theoretically show how we can tune a symmetry transition from a SU(4) ground state, a many body state that forms a spin as well as orbital singlet by virtual exchange with the leads, to a pure SU(2) orbital ground state, as a function of magnetic field. The small size and the s-like orbital symmetry of the ground state of the dopant, make it a model system in which the magnetic field only couples to the spin degree of freedom and allows for observation of this SU(4) to SU(2) transition.