Using a combination of continuous wave and time-resolved spectroscopy, we study the effects of interfacial conditions on the radiative lifetimes and photoluminescence intensities of colloidal CdTe/CdS quantum dots (QDs) embedded in a three-dimensional nanostructured silicon (NSi) matrix. The NSi matrix was thermally oxidized under different conditions to change the interfacial oxide thickness. QDs embedded in a NSi matrix with ~0.5 nm of interfacial oxide exhibited reduced photoluminescence intensity and nearly five times shorter radiative lifetimes (~16 ns) compared to QDs immobilized within completely oxidized, nanostructured silica (NSiO2) frameworks (~78 ns). Optical absorption by the sub-nm oxidized NSi matrix partially lowers QD emission intensities while non-radiative carrier recombination and phonon assisted transitions influenced by defect sites within the oxide and NSi are believed to be the primary factors limiting the QD exciton lifetimes in the heterostructures.