This work shows an unconventional route for spin-driven ferroelectricity originating from a metastable magnetic field-induced canting of chromium sublattice in the presence of gadolinium moments in GdCrTiO5 at low temperatures. Compared to the isostructural neodymium compound, significant differences of magnetism and magnetoelectric effects are seen. We present the results of thorough investigations of temperature and magnetic field dependent magnetization as well as ac and dc magnetic susceptibility. These bulk measurements are complemented by local-probe spectroscopy utilizing electron-spin resonance and muon-spin rotation/relaxation for probing the chromium moments. Ferroelectric order is inferred from pyro- and magnetocurrent measurements. GdCrTiO5 shows a pyrocurrent signal around 10 K, only if the system is cooled in an applied magnetic field exceeding 10 kOe. A distinct spin-driven ferroelectric order is revealed in this state for temperatures below 10 K, which can be switched by changing magnetic-field direction and the polarity of the electric field. But, the magnetic measurements reveal no clear signature of long-range magnetic ordering. The presence of such meta-magnetoelectric-type behaviour in the absence of any meta-magnetic behavior is rare in the literature. Our microscopic spectroscopy results indicate significant changes of the magnetic properties around 10 K. Probably there is an exchange frustration between Gd and Cr moments, which prevents long-range magnetic ordering at further high temperature. Below 10 K, weak magnetic ordering occurs by minimizing frustration due to lattice distortion, which helps in magnetodielectric coupling. However, the non-polar distortion attains appreciable values after application of magnetic fields above 10 kOe to break the spatial inversion symmetry, which creates ferroelectricity.