Using resonant inelastic x-ray scattering, we observe in the bilayer iridate Sr3Ir2O7, a spin-orbit coupling driven magnetic insulator with a small charge gap, a magnon gap of ~92 meV for both acoustic and optical branches. This exceptionally large m
agnon gap exceeds the total magnon bandwidth of ~70 meV and implies a marked departure from the Heisenberg model, in stark contrast to the case of the single-layer iridate Sr2IrO4. Analyzing the origin of these observations, we find that the giant magnon gap results from bond-directional pseudo-dipolar interactions that are strongly enhanced near the metal-insulator transition boundary. This suggests that novel magnetism, such as that inspired by the Kitaev model built on the pseudo-dipolar interactions, may emerge in small charge-gap iridates.
Using resonant x-ray diffraction, we observe an easy c-axis collinear antiferromagnetic structure for the bilayer Sr$_3$Ir$_2$O$_7$, a significant contrast to the single layer Sr$_2$IrO$_4$ with in-plane canted moments. Based on a microscopic model H
amiltonian, we show that the observed spin-flop transition as a function of number of IrO$_2$ layers is due to strong competition among intra- and inter-layer bond-directional pseudo-dipolar interactions of the spin-orbit entangled $J_{eff}$=1/2 moments. With this we unravel the origin of anisotropic exchange interactions in a Mott insulator in the strong spin-orbit coupling regime, which holds the key to the various types of unconventional magnetism proposed in 5$d$ transition metal oxides.
