Observations of gamma ray bursts (GRBs) with Swift produced the initially surprising result that many bursts have large X-ray flares superimposed on the underlying afterglow. The flares were sometimes intense, had rapid rise and decay phases, and occurred late relative to the ``prompt phase. Some remarkable flares are observed with fluence comparable to the prompt GRB fluence. Many GRBs have several flares, which are sometimes overlapping. Short, intense, repetitive, and late flaring can be most easily understood within the context of the standard fireball model with the internal engine that powers the prompt GRB emission in an active state at late times. However, other models for flares have been proposed. Flare origin can be investigated by comparing the flare spectra to that of the afterglow and the initial prompt emission. In this work, we have analyzed all significant X-ray flares from the first 110 GRBs observed by Swift. From this sample 33 GRBs were found to have significant X-ray flares, with 77 flares that were detected above the 3$sigma$ level. In addition to temporal analysis presented in a companion paper, a variety of spectral models have been fit to each flare. In some cases, we find that the spectral fits favor a Band function model, which is more akin to the prompt emission than to that of an afterglow. We find that the average fluence of the flares is 2.4e-7 erg/cm^2/s in the 0.2-10 keV energy band, which is approximately a factor of ten below the average prompt GRB fluence. These results, when combined with those presented in the companion paper on temporal properties of flares, supports the hypothesis that most X-ray flares are late-time activity of the internal engine that spawned the initial GRB; not an afterglow related effect.