On the Evidence for Cosmic Variation of the Fine Structure Constant (I): A Parametric Bayesian Model Selection Analysis of the Quasar Dataset

We review the evidence behind recent claims of spatial variation in the fine structure constant deriving from observations of ionic absorption lines in the light from distant quasars. To this end we expand upon previous non-Bayesian analyses limited by the assumptions of an unbiased and strictly Normal distribution for the unexplained errors of the benchmark quasar dataset. Through the technique of reverse logistic regression we estimate and compare marginal likelihoods for three competing hypotheses---(i) the null hypothesis (no cosmic variation), (ii) the monopole hypothesis (a constant Earth-to-quasar offset), and (iii) the monopole+dipole hypothesis (a cosmic variation manifest to the Earth-bound observer as a North-South divergence)---under a variety of candidate parametric forms for the unexplained error term. Our analysis reveals weak support for a skeptical interpretation in which the apparent dipole effect is driven solely by systematic errors of opposing sign inherent in measurements from the two telescopes employed to obtain these observations. Throughout we seek to exemplify a best practice approach to Bayesian model selection with prior-sensitivity analysis; in a companion paper we extend this methodology to a semi-parametric framework using the infinite-dimensional Dirichlet process.

On the Estimation of Confidence Intervals for Binomial Population Proportions in Astronomy: The Simplicity and Superiority of the Bayesian Approach

I present a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small-to-intermediate samples. Population proportions arise frequently as quantities of interest in astronomical research; for instance, in studies aiming to constrain the bar fraction, AGN fraction, SMBH fraction, merger fraction, or red sequence fraction from counts of galaxies exhibiting distinct morphological features or stellar populations. However, two of the most widely-used techniques for estimating binomial confidence intervals--the normal approximation and the Clopper & Pearson approach--are liable to misrepresent the degree of statistical uncertainty present under sampling conditions routinely encountered in astronomical surveys, leading to an ineffective use of the experimental data (and, worse, an inefficient use of the resources expended in obtaining that data). Hence, I provide here an overview of the fundamentals of binomial statistics with two principal aims: (i) to reveal the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g. R, matlab, mathematica, IDL, python); and (ii) to demonstrate convincingly the major flaws of both the normal approximation and the Clopper & Pearson approach for error estimation.