In the superconducting regime of FeTe$_{(1-x)}$Se$_x$, there exist two types of vortices which are distinct by the presence or absence of zero energy states in their core. To understand their origin, we examine the interplay of Zeeman coupling and superconducting pairings in three-dimensional metals with band inversion. Weak Zeeman fields are found to suppress the intra-orbital spin-singlet pairing, known to localize the states at the ends of the vortices on the surface. On the other hand, an orbital-triplet pairing is shown to be stable against Zeeman interactions, but leads to delocalized zero-energy Majorana modes which extend through the vortex. In contrast, the finite-energy vortex modes remain localized at the vortex ends even when the pairing is of orbital-triplet form. Phenomenologically, this manifests as an observed disappearance of zero-bias peaks within the cores of topological vortices upon increase of the applied magnetic field. The presence of magnetic impurities in FeTe$_{(1-x)}$Se$_x$, which are attracted to the vortices, would lead to such Zeeman-induced delocalization of Majorana modes in a fraction of vortices that capture a large enough number of magnetic impurities. Our results provide an explanation to the dichotomy between topological and non-topological vortices recently observed in FeTe$_{(1-x)}$Se$_x$.