An analysis of neutron and proton scattering off $^{40,48}$Ca has been carried out. Real and imaginary potentials have been generated using the Nuclear Structure Method (NSM) for scattering with the Gogny D1S nucleon-nucleon effective interaction. Observables are well described by NSM for neutron and proton elastic scattering off $^{40}$Ca and for neutron scattering off $^{48}$Ca. For proton scattering off $^{48}$Ca, NSM yields a lack of absorption. This discrepancy is attributed to double-charge-exchange contribution and coupling to Gamow- Teller mode which are not included in the present version of NSM. A recipe based on a Perey-Buck fit of NSM imaginary potential and Lane model is proposed to overcome this issue in an approximate way.