The behavior of conduction electrons on magnetic structures has been intensely investigated. A typical example is the anomalous Hall effect in a ferromagnet. However, distinguishing various anomalous and normal Hall signals induced from the time-reversal symmetry (TRS) broken by their magnetic structure or applied magnetic field is delicate. In this study, we present a method to investigate TRS broken by the magnetic structure by analyzing magnetic quantum oscillations (MQOs). As is known, if a material is nonmagnetic, the MQO phases can only be two distinct values of 0 or $pi$ from the orbits. When the magnetic structure breaks the TRS, the MQO phase deviates from these values, and the deviation is called the anomalous phase. We observed the anomalous phase in Fe-doped NbSb2, where magnetic Fe impurities break the TRS. The phase of a high-doped sample largely deviates from the phases of low-doped and pristine samples, indicating the anomalous phase. In MQOs, different types of magnetic structures afford different field dependence to the phase; this makes it easy to discern different magnetic structures, which respond differently with magnetic fields. This method can complement the Hall measurement and will provide useful information by itself for studying the magnetic structure of materials.