We demonstrate that small but finite ferroelectric polarization ($sim$0.01 $mu$C/cm$^2$) emerges in orthorhombic LuFeO$_3$ ($Pnma$) at $T_N$ ($sim$600 K) because of commensurate (k = 0) and collinear magnetic structure. The synchrotron x-ray and neutron diffraction data suggest that the polarization could originate from enhanced bond covalency together with subtle contribution from lattice. The theoretical calculations indicate enhancement of bond covalency as well as the possibility of structural transition to the polar $Pna2_1$ phase below $T_N$. The $Pna2_1$ phase, in fact, is found to be energetically favorable below $T_N$ in orthorhombic LuFeO$_3$ ($albeit$ with very small energy difference) than in isostructural and nonferroelectric LaFeO$_3$ or NdFeO$_3$. Application of electric field induces finite piezostriction in LuFeO$_3$ via electrostriction resulting in clear domain contrast images in piezoresponse force microscopy.