We present the analysis of a sample of twenty-four SLACS-like galaxy-galaxy strong gravitational lens systems with a background source and deflectors from the Illustris-1 simulation. We study the degeneracy between the complex mass distribution of the lenses, substructures, the surface brightness distribution of the sources, and the time delays. Using a novel inference framework based on Approximate Bayesian Computation, we find that for all the considered lens systems, an elliptical and cored power-law mass density distribution provides a good fit to the data. However, the presence of cores in the simulated lenses affects most reconstructions in the form of a Source Position Transformation. The latter leads to a systematic underestimation of the source sizes by 50 per cent on average, and a fractional error in $H_{0}$ of around $25_{-19}^{+37}$ per cent. The analysis of a control sample of twenty-four lens systems, for which we have perfect knowledge about the shape of the lensing potential, leads to a fractional error on $H_{0}$ of $12_{-3}^{+6}$ per cent. We find no degeneracy between complexity in the lensing potential and the inferred amount of substructures. We recover an average total projected mass fraction in substructures of $f_{rm sub}<1.7-2.0times10^{-3}$ at the 68 per cent confidence level in agreement with zero and the fact that all substructures had been removed from the simulation. Our work highlights the need for higher-resolution simulations to quantify the lensing effect of more realistic galactic potentials better, and that additional observational constraint may be required to break existing degeneracies.