We systematically investigate the ground state and elementary excitations of a Bose-Einstein Condensate with a synthetic vector potential, which is induced by the many-body effects and atom-light coupling. For a sufficiently strong inter-atom interaction, we find the condensate undergoes a Stoner-type ferromagnetic transition through the self-consistent coupling with the vector potential. For a weak interaction, the critical velocity of a supercurrent is found anisotropic due to the density fluctuations affecting the gauge field. We further analytically demonstrate the topological ground state with a coreless vortex ring in a 3D harmonic trap and a coreless vortex-antivortex pair in a 2D trap. The circulating persistent current is measurable in the time-of-flight experiment or in the dipolar oscillation through the violation of Kohn theorem.