The origin of color gradients in elliptical galaxies is examined by comparing model gradients with those observed in the Hubble Deep Field. The models are constructed so as to reproduce color gradients in local elliptical galaxies either by a metallicity gradient or by an age gradient. By looking back a sequence of color gradient as a function of redshift, the age - metallicity degeneracy is solved. The observed color gradients in elliptical galaxies at $z=0.1-1.0$ agree excellently with those predicted by the metallicity gradient, while they deviates significantly from those predicted by the age gradient even at $z sim 0.3$ and the deviation is getting larger with increasing redshift. This result does not depend on cosmological parameters and parameters for an evolutionary model of galaxy within a reasonable range. Thus our results clearly indicate that the origin of color gradients is not the age but the stellar metallicity.