Capturing the non-collinear magnetic ground state of the spinel vanadates AV$_2$O$_4$ (A= Mn, Fe and Co) remains an outstanding challenge for state-of-the-art ab-initio methods. We demonstrate that both the non-collinear spin texture, as well as the magnitude of local moments, are captured by a single value of the on-site Hubbard $U$ of 2.7~eV in conjunction with the local spin density approximation (LSDA+$U$), provided the source term (i.e., magnetic monopole term) is removed from the exchange-correlation magnetic field ${bf B}_{XC}$. We further demonstrate that the magnetic monopole structure in ${bf B}_{XC}$ is highly sensitive to the value of $U$, to the extent that the interplay between on-site localization and local moment magnitude is qualitatively different depending on whether the source term is removed or not. This suggests that in treating strongly correlated magnetic materials within the LSDA+$U$ formalism, subtraction of the unphysical magnetic monopole term from the exchange-correlation magnetic field is essential to correctly treat the magnetic ground state.