V-doped (Bi,Sb)$_2$Te$_3$ has a ten times higher magnetic coercivity than its Cr-doped counterpart and therefore is believed to be a superior system for the quantum anomalous Hall effect (QAHE). The QAHE requires the opening of a magnetic band gap at the Dirac point. We do not find this gap by angle-resolved photoelectron spectroscopy down to 1 K. By x-ray magnetic circular dichroism (XMCD) we directly probe the magnetism at the V site and in zerofield. Hysteresis curves of the XMCD signal show a strong dependence of the coercivity on the ramping velocity of the magnetic field. The XMCD signal decays on a time scale of minutes which we conclude contributes to the absence of a detectable magnetic gap at the Dirac point.