Magneto-optical properties of the ferromagnetic semiconductor GaMnAs are studied in a material specific multi-band tight-binding approach. Two realistic models are compared: one has no impurity band while the other shows an impurity band for low Mn concentrations. The calculated magnetic circular dichroism (MCD) is positive for both models proving that, unlike previously asserted, the observed positive MCD signal is inconclusive as to the presence or absence of an impurity band in GaMnAs. The positive MCD is due to the antiferromagnetic p-d coupling and the transitions into the conduction band.