We report on the micro-photoluminescence spectroscopy of InAs/GaAs quantum dots (QD) doped by a single Mn atom in a magnetic field either longitudinal or perpendicular to the optical axis. In both cases the spectral features of positive trion (X+) are found to split into strongly circularly polarized components, an effect very surprising in a perpendicular magnetic field. The field-induced splitting is ascribed to the transverse Zeeman splitting of the neutral acceptor complex A0 issued by the Mn impurity, whereas the circular optical selection rules result from the p-d exchange which acts as a very strong longitudinal magnetic field inhibiting the spin mixing by the transverse field of the QD heavy-hole ground state. A theoretical model of the spin interactions which includes (i) the local strain anisotropy experienced by the acceptor level and (ii) the anisotropic exchange due to the out-of-center Mn position provides a very good agreement with our observations.