Magnetite, Fe$_3$O$_4$, displays a highly complex low temperature crystal structure that may be charge and orbitally ordered. Many of the recent experimental claims of such ordering rely on resonant soft x-ray diffraction at the oxygen K and iron L edges. We have re-examined this system and undertaken soft x-ray diffraction experiments on a high-quality single crystal. Contrary to previous claims in the literature, we show that the intensity observed at the Bragg forbidden (001/2)$_c$ reflection can be explained purely in terms of the low-temperature structural displacements around the resonant atoms. This does not necessarily mean that magnetite is not charge or orbitally ordered, but rather that the present sensitivity of resonant soft x-ray experiments does not allow conclusive demonstration of such ordering.