We have recently reported that the Haldane spin-chain system, Er2BaNiO5, undergoing antiferromagnetic order below 32 K, is characterized by the onset of ferroelectricity near 60K due to magnetoelectric coupling induced by short-range magnetic-order within spin-chains. We have carried out additional magnetic and dielectric studies to understand the properties well below antiferromagnetic ordering temperature. We emphasize here on the following: (i) A strong frequency dependent behaviors of ac magnetic susceptibility and complex dielectric properties have been observed at much lower temperatures (below 8 K), that is, reentrant multiglass-like phenomenon, naturally suggesting the existence of an additional transition well below Neel temperature; ii) Magnetoelectric phase coexistence is observed at very low temperature (e.g., T =2K), where the high-field magnetoelectric phase is partially arrested on returning to zero magnetic field after a cycling through metamagnetic transition.