We report UBVRI observations of the optical afterglow of the gamma-ray burst GRB 021004. We observed significant (10-20%) deviations from a power law decay on several time scales, ranging from a few hours down to 20-30 minutes. We also observed a significant color change starting ~1.5 days after the burst, confirming the spectroscopic results already reported by Matheson et al. (2002). We discuss these results in the context of several models that have recently been proposed to account for the anomalous photometric behavior of this event.