How to control collectively ordered electronic states is a core interest of condensed matter physics. We report an electric field controlled reversible transition from superconductor to ferromagnetic insulator in (Li,Fe)OHFeSe thin flake using solid ion conductor as the gate dielectric. By driving Li ions into and out of the (Li,Fe)OHFeSe thin flake with electric field, we obtained a dome-shaped superconducting region with optimal Tc ~ 43 K, which is separated by a quantum critical point from ferromagnetically insulating phase. The ferromagnetism arises from the long range order of the interstitial Fe ions expelled from the (Li,Fe)OH layers by Li injection. The device can reversibly manipulate collectively ordered electronic states and stabilize new metastable structures by electric field.