The structural and magnetic properties of ultrathin FeO(111) films on Pt(111) with thicknesses from 1 to 16 monolayers (ML) were studied using the nuclear inelastic scattering (NIS) of synchrotron radiation. Distinct evolution of vibrational characteristics with thickness that is revealed in the phonon density of states (PDOS) witnesses a textbook transition from 2D to 3D lattice dynamics. For the thinnest films of 1 and 2 ML, the low energy part of the PDOS followed a linear dependence in energy that is characteristic for 2-dimensional systems. This dependence gradually transforms with thickness to the bulk ~E-square relation. Density functional theory phonon calculations perfectly reproduced the measured 1 ML PDOS within a simple model of a pseudomorphic FeO/Pt(111) interface. The calculations show that the 2D PDOS character is due to a weak coupling of the FeO film to the Pt(111) substrate. The evolution of the vibrational properties with an increasing thickness is closely related to a transient long range magnetic order and stabilization of an unusual structural phase.