Sub-gap states in semiconducting-superconducting nanowire hybrid devices are controversially discussed as potential topologically non-trivial quantum states. One source of ambiguity is the lack of an energetically and spatially well defined tunnel spectrometer. Here, we use quantum dots directly integrated into the nanowire during the growth process to perform tunnel spectroscopy of discrete sub-gap states in a long nanowire segment. In addition to sub-gap states with a standard magnetic field dependence, we find topologically trivial sub-gap states that are independent of the external magnetic field, i.e. that are pinned to a constant energy as a function of field. We explain this effect qualitatively and quantitatively by taking into account the strong spin-orbit interaction in the nanowire, which can lead to a decoupling of Andreev bound states from the field due to a spatial spin texture of the confined eigenstates.