From a sample of GRBs detected by the $Fermi$ and $Swift$ missions, we have extracted the minimum variability time scales for temporal structures in the light curves associated with the prompt emission and X-ray flares. A comparison of this variability time scale with pulse parameters such as rise times,determined via pulse-fitting procedures, and spectral lags, extracted via the cross-correlation function (CCF), indicate a tight correlation between these temporal features for both the X-ray flares and the prompt emission. These correlations suggests a common origin for the production of X-ray flares and the prompt emission in GRBs.