We explore how the growth rate of spiral disks can be measured via analyses of the scatter in the Tully-Fisher (TF) relation of local and intermediate redshift galaxies. As an initial step, we show it is possible to construct a low-dispersion TF relation for nearly face-on, nearby spirals. We find these spiral disks are non-circular, (a mean ellipticity of 6%), which accounts for about 0.1 mag of the intrinsic scatter in the nearby TF relation of even the most ``normal looking spirals. If this ellipticity is induced by matter accretion, we expect to find greater disk ellipticity or disturbance in the past. We find evidence that more extreme outliers of the intermediate-redshift TF relation are more morphologically and kinematically disturbed. Whether this effect reflects a redshift trend or selection bias of local samples needs to be addressed.