Single monolayer FeSe film grown on Nb-doped SrTiO$_3$(001) substrate shows the highest superconducting transition temperature (T$_C$ $sim$ 100 K) among the iron-based superconductors (iron-pnictide), while T$_C$ of bulk FeSe is only $sim$ 8 K. Antiferromagnetic spin fluctuations were believed to be crucial in iron-pnictides, which has inspired several proposals to understand the FeSe/SrTiO$_3$ system. Although bulk FeSe does not show the antiferromagnetic order, calculations suggest that the parent FeSe/SrTiO$_3$ films are AFM. Experimentally, due to lacking of direct probe, the magnetic state of FeSe/SrTiO$_3$ films remains mysterious. Here, we report the direct evidences of the antiferromagnetic order in the parent FeSe/SrTiO$_3$ films by the magnetic exchange bias effect measurements. The phase transition temperature is $geq$ 140 K for single monolayer film. The AFM order disappears after electron doping.