Hybrid qubit systems combining electronic spins with nearby (proximate) nuclear spin registers offer a promising avenue towards quantum information processing, with even multi-spin error correction protocols recently demonstrated in diamond. However, for the important platform offered by spins of donor atoms in cryogenically-cooled silicon,decoherence mechanisms of $^{29}$Si proximate nuclear spins are not yet well understood.The reason is partly because proximate spins lie within a so-called frozen core region where the donor electronic hyperfine interaction strongly suppresses nuclear dynamics. We investigate the decoherence of a central proximate nuclear qubit arising from quantum spin baths outside, as well as inside, the frozen core around the donor electron. We consider the effect of a very large nuclear spin bath comprising many ($gtrsim 10^8$) weakly contributing pairs outside the frozen core. We also propose that there may be an important contribution from a few (of order $100$) symmetrically sited nuclear spin pairs (equivalent pairs), which were not previously considered as their effect is negligible outside the frozen core. If equivalent pairs represent a measurable source of decoherence, nuclear coherence decays could provide sensitive probes of the symmetries of electronic wavefunctions. For the phosphorus donor system, we obtain $T_{2n}$ values of order 1 second for both the far bath and equivalent pair models, confirming the suitability of proximate nuclei in silicon as very long-lived spin qubits.