The layered perovskite compounds are interesting due to their intriguing physical properties. In this article we report the structural, magnetic and dielectric properties of LnBaCuFeO5 (Ln=Nd, Eu, Gd, Ho and Yb). The structural parameters decrease from Nd to Yb due to the decrease in the ionic radii of the rare earth ions. An antiferromagnetic transition is observed for EuBaCuFeO5 near 120 K along with the glassy dynamics of the electric dipoles below 100 K. The magnetic transition is absent in other compounds, which may be due to the dominance of the magnetic moment of the rare earth ions. The dielectric constant does not show any anomaly, except in the case of HoBaCuFeO5 where it shows a weak frequency dependence around 54 K. These compounds show a significant enhancement of dielectric constant at high temperatures which have been attributed to Maxwell-Wagner effect. However, no significant magneto-dielectric coupling has been observed in these layered perovskites.