The optical spectroscopy of a single InAs quantum dot doped with a single Mn atom is studied using a model Hamiltonian that includes the exchange interactions between the spins of the quantum dot electron-hole pair, the Mn atom and the acceptor hole. Our model permits to link the photoluminescence spectra to the Mn spin states after photon emission. We focus on the relation between the charge state of the Mn, $A^0$ or $A^-$, and the different spectra which result through either band-to-band or band-to-acceptor transitions. We consider both neutral and negatively charged dots. Our model is able to account for recent experimental results on single Mn doped InAs PL spectra and can be used to account for future experiments in GaAs quantum dots. Similarities and differences with the case of single Mn doped CdTe quantum dots are discussed.