Ternary iron arsenide EuFe$_2$As$_2$ with ThCr$_2$Si$_2$-type structure has been studied by magnetic susceptibility, resistivity, thermopower, Hall and specific heat measurements. The compound undergoes two magnetic phase transitions at about 200 K and 20 K, respectively. The former was found to be accompanied with a slight drop in magnetic susceptibility (after subtracting the Curie-Weiss paramagnetic contribution), a rapid decrease in resistivity, a large jump in thermopower and a sharp peak in specific heat with decreasing temperature, all of which point to a spin-density-wave-like antiferromagnetic transition. The latter was proposed to be associated with an A-type antiferromagnetic ordering of Eu$^{2+}$ moments. Comparing with the physical properties of the iso-structural compounds BaFe$_2$As$_2$ and SrFe$_2$As$_2$, we expect that superconductivity could be induced in EuFe$_2$As$_2$ through appropriate doping.