We present the age distributions for star clusters and individual stars in the Small Magellanic Cloud (SMC) based on data from the Magellanic Clouds Photometric Survey by Zaritsky and collaborators. The age distribution of the SMC clusters shows a steep decline, dN_{cluster}/dt propto t^{-0.85pm0.15}, over the period 10^7 < t <10^9 yr. This decline is essentially identical to that observed previously for more massive clusters in the merging Antennae galaxies, and also for lower-mass embedded clusters in the solar neighborhood. The SMC cluster age distribution therefore provides additional evidence for the rapid disruption of star clusters (``infant mortality). These disrupted clusters deliver their stars to the general field population, implying that the field star age distribution, dN_{fld star}/dt, should have an inverse relation to dN_{cluster}/dt if most stars form initially in clusters. We make specific predictions for dN_{fldstar}/dt based on our cluster disruption models, and compare them with current data available for stars in the SMC. While these data do not extend to sufficiently young ages for a definitive test, they are consistent with a scenario wherein most SMC stars formed in clusters. Future analyses of dN_{fldstar}/dt that extend down to ages of approximately few million years are needed to verify the age relationship between stars residing in clusters and in the field.