We report temperature-dependent polarized optical conductivity [$sigma(omega)$] spectra of CeFe$_2$Al$_{10}$, which is a reference material for CeRu$_2$Al$_{10}$ and CeOs$_2$Al$_{10}$ with an anomalous magnetic transition at 28 K. The $sigma(omega)$ spectrum along the b-axis differs greatly from that in the $ac$-plane, indicating that this material has an anisotropic electronic structure. At low temperatures, in all axes, a shoulder structure due to the optical transition across the hybridization gap between the conduction band and the localized $4f$ states, namely $c$-$f$ hybridization, appears at 55 meV. However, the gap opening temperature and the temperature of appearance of the quasiparticle Drude weight are strongly anisotropic indicating the anisotropic Kondo temperature. The strong anisotropic nature in both electronic structure and Kondo temperature is considered to be relevant the anomalous magnetic phase transition in CeRu$_2$Al$_{10}$ and CeOs$_2$Al$_{10}$.