(abridged) An outburst of more than 80 individual bursts, similar to those seen from Soft Gamma Repeaters (SGRs), was detected from the Anomalous X-ray Pulsar (AXP) 1E 2259+586 in 2002 June. Coincident with this burst activity were gross changes in the pulsed flux, persistent flux, energy spectrum, pulse profile and spin down of the underlying X-ray source. We present RXTE and XMM-Newton observations of 1E 2259+586 that show the evolution of the aforementioned source parameters during and following this episode. Specifically, we observe an X-ray flux increase by more than an order of magnitude having two distinct components. The first component is linked to the burst activity and decays within ~2 days during which the energy spectrum is considerably harder than during the quiescent state of the source. The second component decays over the year following the glitch according to a power law in time with an exponent -0.22 +/- 0.01. The pulsed fraction decreased initially to ~15% RMS, and the pulse profile changed significantly during the outburst. A glitch was observed in 1E 2259+586 that preceded the observed burst activity. A fraction of the glitch (~19%) recovered, although the recovery was not purely exponential. An exponential rise of ~20% of the frequency jump with a time scale of ~14 days results in a significantly better fit to the data, however, contamination from a systematic drift in the phase of the pulse profile cannot be excluded. The long-term post-glitch spin-down rate decreased in magnitude relative to the pre-glitch value. A comparison with SGR outburst properties, a physical interpretation of the results, and implications on the number of magnetar candidates in our Galaxy are discussed.