We study the ``normal decay phase of the X-ray afterglows of gamma-ray bursts (GRBs), which follows the shallow decay phase, using the events simultaneously observed in the R-band. The classical external shock model -- in which neither the delayed energy injection nor time-dependency of shock micro-physics is considered -- shows that the decay indices of the X-ray and R-band light curves, $alpha_{rm X}$ and $alpha_{rm O}$, obey a certain relation, and that in particular, $alpha_{rm O}-alpha_{rm X}$ should be larger than -1/4 unless the ambient density increases with the distance from the central engine. For our selected 14 samples, we have found that 4 events violate the limit at more than the 3$sigma$ level, so that a fraction of events are outliers of the classical external shock model at the ``normal decay phase.