Anomalies drive scientific discovery -- they are associated with the cutting edge of the research frontier, and thus typically exploit data in the low signal-to-noise regime. In astronomy, the prevalence of systematics --- both known unknowns and unknown unknowns --- combined with increasingly large datasets, the widespread use of ad hoc estimators for anomaly detection, and the look-elsewhere effect, can lead to spurious false detections. In this informal note, I argue that anomaly detection leading to discoveries of new physics requires a combination of physical understanding, careful experimental design to avoid confirmation bias, and self-consistent statistical methods. These points are illustrated with several concrete examples from cosmology.
تحميل البحث