The radiative and non-radiative decay rates of InAs quantum dots are measured by controlling the local density of optical states near an interface. From time-resolved measurements we extract the oscillator strength and the quantum efficiency and their dependence on emission energy. From our results and a theoretical model we determine the striking dependence of the overlap of the electron and hole wavefunctions on the quantum dot size. We conclude that the optical quality is best for large quantum dots, which is important in order to optimally tailor quantum dot emitters for, e.g., quantum electrodynamics experiments.