Time resolved intensity cross-correlation measurements of radiative cascades are used for studying non-radiative relaxation processes of excited carriers confined in semiconductor quantum dots. We spectrally identify indirect radiative cascades which include intermediate phonon assisted relaxations. The energy of the first photon reveals the multicarrier configuration prior to the non-radiative relaxation, while the energy of the second photon reveals the configuration after the relaxation. The intensity cross correlation measurements thus provide quantitative measures of the non-radiative processes and their selection rules. We construct a model which accurately describes the experimental observations in terms of the electron-phonon and electron-hole exchange interactions. Our measurements and model provide a new tool for engineering relaxation processes in semiconductor nanostructures.