By the first-principles electronic structure calculations, we find that the ground state of PbO-type tetragonal $alpha$-FeTe is in a bi-collinear antiferromagnetic state, in which the Fe local moments ($sim2.5mu_B$) are ordered ferromagnetically along a diagonal direction and antiferromagnetically along the other diagonal direction on the Fe square lattice. This bi-collinear order results from the interplay among the nearest, next nearest, and next next nearest neighbor superexchange interactions $J_1$, $J_2$, and $J_3$, mediated by Te $5p$-band. In contrast, the ground state of $alpha$-FeSe is in the collinear antiferromagnetic order, similar as in LaFeAsO and BaFe$_2$As$_2$.