We present a detailed study of magnetism in LuFe2O4, combining magnetization measurements with neutron and soft x-ray diffraction. The magnetic phase diagram in the vicinity of T_N involves a metamagnetic transition separating an antiferro- and a ferrimagnetic phase. For both phases the spin structure is refined by neutron diffraction. Observed diffuse magnetic scattering far above T_N is explained in terms of near degeneracy of the magnetic phases.