In 2004, SGR 1806-20 underwent a period of intense and long-lasting burst activity that included the giant flare of 27 December 2004 -- the most intense extra-solar transient event ever detected at Earth. During this active episode, we routinely monitored the source with Rossi X-ray Timing Explorer and occasionally with Chandra. During the course of these observations, we identified two relatively bright bursts observed with Konus-Wind in hard X-rays that were followed by extended X-ray tails or afterglows lasting hundreds to thousands of seconds. Here, we present detailed spectral and temporal analysis of these events observed about 6 and 1.5 months prior to the 27 December 2004 Giant Flare. We find that both X-ray tails are consistent with a cooling blackbody of constant radius. These spectral results are qualitatively similar to those of the burst afterglows recorded from SGR 1900+14 and recently from SGR 1550-5418. However, the latter two sources exhibit significant increase in their pulsed X-ray intensity following the burst, while we did not detect any significant changes in the RMS pulsed amplitude during the SGR 1806-20 events. Moreover, we find that the fraction of energy partitioned to the burst (prompt energy release) and the tail (afterglow) differs by an order of magnitude between SGR 1900+14 and SGR 1806-20. We suggest that such differences can be attributed to differences in the crustal heating mechanism of these neutron stars combined with the geometry of the emitting areas.