Magneto-capacitance effect was investigated using the impedance spectroscopy on single crystals of LuFe2O4. The intrinsic impedance response could be separated from the interfacial response and showed a clear hysteresis loop below TFerri ~ 240 K under the magnetic field. The neutron diffraction experiment under the magnetic field proves the origin of dielectric property related to the motion of nano-sized ferromagnetic domain boundary. These results imply that the modification of the microscopic domain structure is responsible for the magnetoelectric effect in LuFe2O4.