On a Rayleigh-Faber-Krahn inequality for the regional fractional Laplacian

We study a Rayleigh-Faber-Krahn inequality for regional fractional Laplacian operators. In particular, we show that there exists a compactly supported nonnegative Sobolev function $u_0$ that attains the infimum (which will be a positive real number) of the set [ left{ intint_{{u > 0}times{u>0}} frac{|u(x) - u(y)|^2}{|x - y|^{n + 2 sigma}}d x d y : u in mathring H^sigma(mathbb{R}^n), int_{mathbb{R}^n} u^2 = 1, |{u > 0 }| leq 1right}. ] Unlike the corresponding problem for the usual fractional Laplacian, where the domain of the integration is $mathbb{R}^n times mathbb{R}^n$, symmetrization techniques may not apply here. Our approach is instead based on the direct method and new a priori diameter estimates. We also present several remaining open questions concerning the regularity and shape of the minimizers, and the form of the Euler-Lagrange equations.