The extreme mass ratio inspiral (EMRI), defined as a stellar-mass compact object inspiraling into a supermassive black hole (SMBH), has been widely argued to be a low-frequency gravitational wave (GW) source. EMRIs providing accurate measurements of black hole mass and spin, are one of the primary interests for Laser Interferometer Space Antenna (LISA). However, it is usually believed that there are no electromagnetic (EM) counterparts to EMRIs. Here we show a new formation channel of EMRIs with tidal disruption flares as EM counterparts. In this scenario, flares can be produced from the tidal stripping of the helium (He) envelope of a massive star by an SMBH. The left compact core of the massive star will evolve into an EMRI. We find that, under certain initial eccentricity and semimajor axis, the GW frequency of the inspiral can enter LISA band within 10 $sim$ 20 years, which makes the tidal disruption flare an EM precursor to EMRI. Although the event rate is just $2times 10^{-4}~rm Gpc^{-3}yr^{-1}$, this association can not only improve the localization accuracy of LISA and help to find the host galaxy of EMRI, but also serve as a new GW standard siren for cosmology.