We present the c-axis optical reflectance measurement on single crystals of BaFe$_2$As$_2$ and SrFe$_2$As$_2$, the parent compounds of FeAs based superconductors. Different from the ab-plane optical response where two distinct energy gaps were observed in the SDW state, only the smaller energy gap could be seen clearly for textbf{E}$parallel$c-axis. The very pronounced energy gap structure seen at a higher energy scale for textbf{E}$parallel$ab-plane is almost invisible. We propose a novel picture for the band structure evolution across the SDW transition and suggest different driving mechanisms for the formation of the two energy gaps.