We report for the first time the magnetic structure of the high entropy oxide $(Mg_{0.2}Co_{0.2}Ni_{0.2}Cu_{0.2}Zn_{0.2})O$ using neutron powder diffraction. This material exhibits a sluggish magnetic transition but possesses a long-range ordered antiferromagnetic ground state, as revealed by DC and AC magnetic susceptibility, elastic and inelastic neutron scattering measurements. The magnetic propagation wavevector is k=(1/2, 1/2, 1/2) based on the cubic structure Fm-3m, and the magnetic structure consists of ferromagnetic sheets in the (111) planes with spins antiparallel between two neighboring planes. Inelastic neutron scattering reveals strong magnetic excitations at 100 K that survive up to room temperature. This work demonstrates that entropy-stabilized oxides represent a unique platform to study long range magnetic order with extreme chemical disorder.