Multiferroics, where two or more ferroic order parameters coexist, is one of the hottest fields in condensed matter physics and materials science[1-9]. However, the coexistence of magnetism and conventional ferroelectricity is physically unfavoured[10]. Recently several remedies have been proposed, e.g., improper ferroelectricity induced by specific magnetic[6] or charge orders[2]. Guiding by these theories, currently most research is focused on frustrated magnets, which usually have complicated magnetic structure and low magnetic ordering temperature, consequently far from the practical application. Simple collinear magnets, which can have high magnetic transition temperature, have never been considered seriously as the candidates for multiferroics. Here, we argue that actually simple interatomic magnetic exchange interaction already contains a driving force for ferroelectricity, thus providing a new microscopic mechanism for the coexistence and strong coupling between ferroelectricity and magnetism. We demonstrate this mechanism by showing that even the simplest antiferromagnetic (AFM) insulator MnO, can display a magnetically induced ferroelectricity under a biaxial strain.