We prove exponential decay of correlations for Holder continuous observables with respect to any Gibbs measure for contact Anosov flows admitting Pesin sets with exponentially small tails. This is achieved by establishing strong spectral estimates for certain Ruelle transfer operators for such flows.