Competing interactions and geometric frustration provide favourable conditions for exotic states of matter. Such competition often causes multiple phase transitions as a function of temperature and can lead to magnetic structures that break inversion symmetry, thereby inducing ferroelectricity [1-4]. Although this phenomenon is understood phenomenologically [3-4], it is of great interest to have a conceptually simpler system in which ferroelectricity appears coincident with a single magnetic phase transition. Here we report the first such direct transition from a paramagnetic and paraelectric phase to an incommensurate multiferroic in the triangular lattice antiferromagnet RbFe(MoO4)2 (RFMO). A magnetic field extinguishes the electric polarization when the symmetry of the magnetic order changes and ferroelectricity is only observed when the magnetic structure has chirality and breaks inversion symmetry. Multiferroic behaviour in RFMO provides a theoretically tractable example of ferroelectricity from competing spin interactions. A Landau expansion of symmetry-allowed terms in the free energy demonstrates that the chiral magnetic order of the triangular lattice antiferromagnet gives rise to a pseudoelectric field, whose temperature dependence agrees with that observed experimentally.