We observe the appearance of a phonon near the lock-in temperature in orthorhombic REMnO3 (RE: Lu and Ho) and anomalous phonon hardening in orthorhombic LuMnO3. The anomalous phonon occurs at the onset of spontaneous polarization. No such changes were found in incommensurate orthorhombic DyMnO3. These observations directly reveal different electric polarization mechanisms in the E-type and IC-type REMnO3.