Magnetoelectric materials are attractive for several applications, including actuators, switches, and magnetic field sensors. Typical mechanisms for achieving a strong magnetoelectric coupling are rooted in transition metal magnetism. In sharp contrast, here we identify CsEr(MoO4)2 as a magnetoelectric material without magnetic transition metal ions, thus ensuring that the Er ions play a key role in achieving this interesting property. Our detailed study includes measurements of the structural, magnetic, and electric properties of this material. Bulk characterization and neutron powder diffraction show no evidence for structural phase transitions down to 0.3 K and therefore CsEr(MoO4)2 maintains the room temperature P2/c space group over a wide temperature range without external magnetic field. These same measurements also identify collinear antiferromagnetic ordering of the Er3+ moments below TN = 0.87 K. Complementary dielectric constant and pyroelectric current measurements reveal that a ferroelectric phase (P ~ 0.5 nC/cm2) emerges when applying a modest external magnetic field, which indicates that this material has a strong magnetoelectric coupling. We argue that the magnetoelectric coupling in this system arises from a pseudo Jahn-Teller distortion induced by the magnetic field.