اشترك بالحزمة الذهبية واحصل على وصول غير محدود شمرا أكاديميا
تسجيل مستخدم جديدStatistical Significance Testing in Information Retrieval: An Empirical Analysis of Type I, Type II and Type III Errors
60
0
0.0
(
0
)
اسأل ChatGPT حول البحث
ﻻ يوجد ملخص باللغة العربية
Statistical significance testing is widely accepted as a means to assess how well a difference in effectiveness reflects an actual difference between systems, as opposed to random noise because of the selection of topics. According to recent surveys on SIGIR, CIKM, ECIR and TOIS papers, the t-test is the most popular choice among IR researchers. However, previous work has suggested computer intensive tests like the bootstrap or the permutation test, based mainly on theoretical arguments. On empirical grounds, others have suggested non-parametric alternatives such as the Wilcoxon test. Indeed, the question of which tests we should use has accompanied IR and related fields for decades now. Previous theoretical studies on this matter were limited in that we know that test assumptions are not met in IR experiments, and empirical studies were limited in that we do not have the necessary control over the null hypotheses to compute actual Type I and Type II error rates under realistic conditions. Therefore, not only is it unclear which test to use, but also how much trust we should put in them. In contrast to past studies, in this paper we employ a recent simulation methodology from TREC data to go around these limitations. Our study comprises over 500 million p-values computed for a range of tests, systems, effectiveness measures, topic set sizes and effect sizes, and for both the 2-tail and 1-tail cases. Having such a large supply of IR evaluation data with full knowledge of the null hypotheses, we are finally in a position to evaluate how well statistical significance tests really behave with IR data, and make sound recommendations for practitioners.
قيم البحث
اقرأ أيضاً
We have conducted spectropolarimetry of 12 type II (obscured) quasar candidates selected from the spectroscopic database of the Sloan Digital Sky Survey based on their emission line properties. Polarization was detected in all objects, with nine bein
g highly polarized (> 3%) and with polarization reaching as high as 17% in two objects. Broad lines were detected in the polarized spectra of five objects. These observations prove beyond a reasonable doubt that the objects in our sample are indeed type II quasars, in that they harbor luminous UV-excess AGNs in their centers and that the direct view to the AGN is highly obscured. For three of the objects in this paper, we have obtained HST images in three bands. The HST observations, combined with the spectropolarimetry data, imply that scattering off material outside the obscuration plane is the dominant polarization mechanism. In all three objects the sizes of scattering regions are a few kpc. For one object, the extent of the scattering region, coupled with the characteristics of the polarized spectrum, argue strongly that dust scattering rather than electron scattering dominates the polarized light. Our observations are well-described by the basic orientation-based unification model of toroidal obscuration and off-plane scattering, implying that the model can be extended to include at least some high-luminosity AGNs.
We investigate interacting spin susceptibilities in lattice models for $mathcal{T}$-reversal symmetry-broken Weyl semimetals. We employ a random phase approximation (RPA) method for the spin-SU(2)-symmetry-broken case that includes mixtures of ladder
and bubble diagrams, beyond a SU(2)-symmetric case. Within this approach, the relations between the tendency towards magnetic order and the band structure tilt parameter $gamma$ under different temperatures are explored. The critical interaction strength $U_c$ for magnetic ordering decreases as the tilt term changes from type-I Weyl semimetals to type-II. The lower temperature, the sharper is the drop in $U_c$ at the critical point between them. The variation of $U_c$ with a slight doping near half-filling is also studied. It is generally found that these Weyl systems show a strongly anisotropic spin response with an enhanced doubly degenerate transverse susceptibility perpendicular to tilt direction, inherited from $mathcal{C}_{4z}$ rational symmetry of bare Hamiltonian, but with the longitudinal response suppressed with respect to that. For small tilts $gamma$ and strong enough interaction, we find two degenerate ordering patterns with spin order orthogonal to the tilt direction but much shorter spin correlation length parallel to the spin direction. With increasing the tilt, the system develops instabilities with respect to in-plane magnetic orders with wavevector $(0,pi, q_z)$ and $(pi,0, q_z)$, with $q_z$ increasing from 0 to $pi$ before the transition to a type-II Weyl semimetal is reached. These results indicate a greater richness of magnetic phases in correlated Weyl semimetals that also pose challenges for precise theoretical descriptions.
In recent years several small basaltic V-type asteroids have been identified all around the main belt. Most of them are members of the Vesta dynamical family, but an increasingly large number appear to have no link with it. The question that arises i
s whether all these basaltic objects do indeed come from Vesta. To find the answer to the above questioning, we decided to perform a statistical analysis of the spectroscopic and mineralogical properties of a large sample of V-types, with the objective to highlight similarities and differences among them, and shed light on their unique, or not, origin. The analysis was performed using 190 visible and near-infrared spectra from the literature for 117 V-type asteroids. The asteroids were grouped according to their dynamical properties and their computed spectral parameters compared. Comparison was also performed with spectral parameters of a sample of HED meteorites and data of the surface of Vesta taken by the VIR instrument on board of the Dawn spacecraft. Our analysis shows that although most of the V-type asteroids in the inner main belt do have a surface composition compatible with an origin from Vesta, this seem not to be the case for V-types in the middle and outer main belt.
The type-II Dirac semimetal PdTe2 was recently reported to be a type-I superconductor with a superconducting transition temperature Tc = 1.65 K. However, the recent results from tunneling and point contact spectroscopy suggested the unusual state of
a mixture of type-I and type-II superconductivity. These contradictory results mean that there is no clear picture of the superconducting phase diagram and warrants a detailed investigation of the superconducting phase. We report here the muon spin rotation and relaxation ($mu$SR) measurements on the superconducting state of the topological Dirac semimetal PdTe2. From $mu$SR measurements, we find that PdTe2 exhibits mixed type-I/type-II superconductivity. Using these results a phase diagram has been determined. In contrast to previous results where local type-II superconductivity persists up to Hc2 = 600 G, we observed that bulk superconductivity is destroyed above 225 G.
The type II Dirac semimetal PdTe$_2$ is unique in the family of topological parent materials because it displays a superconducting ground state below 1.7 K. Despite wide speculations on the possibility of an unconventional topological superconducting
phase, tunneling and heat capacity measurements revealed that the superconducting phase of PdTe$_2$ follows predictions of the microscopic theory of Bardeen, Cooper and Shriefer (BCS) for conventional superconductors. The superconducting phase in PdTe$_2$ is further interesting because it also displays properties that are characteristics of type-I superconductors and are generally unexpected for binary compounds. Here, from scanning tunneling spectroscopic measurements we show that the surface of PdTe$_2$ displays intrinsic electronic inhomegenities in the normal state which leads to a mixed type I and type II superconducting behaviour along with a spatial distribution of critical fields in the superconducting state. Understanding of the origin of such inhomogeneities may be important for understanding the topological properties of PdTe$_2$ in the normal state.
الأسئلة المقترحة
سجل دخول لتتمكن من نشر تعليقات
التعليقات
جاري جلب التعليقات
سجل دخول لتتمكن من متابعة معايير البحث التي قمت باختيارها
