The unambiguous observation of a Chiral Magnetic Effect (CME)-driven charge separation is the core aim of the isobar program at RHIC consisting of ${^{96}_{40}}$Zr+${^{96}_{40}}$Zr and ${^{96}_{44}}$Ru+${^{96}_{44}}$Ru collisions at $sqrt {s_{rm NN}}!=!200$ GeV. We quantify the role of the spatial distributions of the nucleons in the isobars on both eccentricity and magnetic field strength within a relativistic hadronic transport approach (SMASH, Simulating Many Accelerated Strongly-interacting Hadrons). In particular, we introduce isospin-dependent nucleon-nucleon spatial correlations in the geometric description of both nuclei, deformation for ${^{96}_{44}}$Ru and the so-called neutron skin effect for the neutron-rich isobar i.e. ${^{96}_{40}}$Zr. The main result of this study is a reduction of the magnetic field strength difference between ${^{96}_{44}}$Ru+${^{96}_{44}}$Ru and ${^{96}_{40}}$Zr+${^{96}_{40}}$Zr by a factor of 2, from $10%$ to $5%$ in peripheral collisions when the neutron-skin effect is included. Further, we find an increase of eccentricity by up to 10$%$ when deformation is taken into account while neither the neutron skin effect nor the nucleon-nucleon correlations result into a significant modification of this observable with respect to the traditional Woods-Saxon modeling. Our results suggest a significantly smaller CME signal to background ratio for the experimental charge separation measurement in peripheral collisions with the isobar systems than previously expected.