These lectures describe several topics in statistical data analysis as used in High Energy Physics. They focus on areas most relevant to analyses at the LHC that search for new physical phenomena, including statistical tests for discovery and exclusi
on limits. Particular attention is payed to the treatment of systematic uncertainties through nuisance parameters.
We present the asymptotic distribution for two-sided tests based on the profile likelihood ratio with lower and upper boundaries on the parameter of interest. This situation is relevant for branching ratios and the elements of unitary matrices such as the CKM matrix.
We propose a method for setting limits that avoids excluding parameter values for which the sensitivity falls below a specified threshold. These power-constrained limits (PCL) address the issue that motivated the widely used CLs procedure, but do so
in a way that makes more transparent the properties of the statistical test to which each value of the parameter is subjected. A case of particular interest is for upper limits on parameters that are proportional to the cross section of a process whose existence is not yet established. The basic idea of the power constraint can easily be applied, however, to other types of limits.
We describe likelihood-based statistical tests for use in high energy physics for the discovery of new phenomena and for construction of confidence intervals on model parameters. We focus on the properties of the test procedures that allow one to acc
ount for systematic uncertainties. Explicit formulae for the asymptotic distributions of test statistics are derived using results of Wilks and Wald. We motivate and justify the use of a representative data set, called the Asimov data set, which provides a simple method to obtain the median experimental sensitivity of a search or measurement as well as fluctuations about this expectation.
