We investigate a geometrically symmetric gold-silicon sphere heterodimer and reveal its extrinsic chiroptical response caused by the interaction with a substrate. The chiroptical response is obtained for oblique incidence owing to the coalescence of extrinsic chirality, heterogeneity and substrate induced break of symmetry. To quantify the chiral response we utilize k-space polarimetry. We elucidate the physics of the involved phenomena by considering scattering properties of the heterodimer in free space and find that incident linearly polarized light is scattered in a spin-split fashion. We corroborate our finding with a coupled dipole model and find that the spin-split behavior originates from the heterogeneity of the structure. This spin-split scattering, combined with the substrate-induced break of symmetry, leads to an extrinsic chiroptical response. Our work sheds new light on the potential and optical properties of heterogeneous nanostructures and paves the way for designing spectrally tunable polarization controlled heterogeneous optical elements.