We introduce an ensemble Markov chain Monte Carlo approach to sampling from a probability density with known likelihood. This method upgrades an underlying Markov chain by allowing an ensemble of such chains to interact via a process in which one cha
ins state is cloned as anothers is deleted. This effective teleportation of states can overcome issues of metastability in the underlying chain, as the scheme enjoys rapid mixing once the modes of the target density have been populated. We derive a mean-field limit for the evolution of the ensemble. We analyze the global and local convergence of this mean-field limit, showing asymptotic convergence independent of the spectral gap of the underlying Markov chain, and moreover we interpret the limiting evolution as a gradient flow. We explain how interaction can be applied selectively to a subset of state variables in order to maintain advantage on very high-dimensional problems. Finally we present the application of our methodology to Bayesian hyperparameter estimation for Gaussian process regression.
Many rare weather events, including hurricanes, droughts, and floods, dramatically impact human life. To accurately forecast these events and characterize their climatology requires specialized mathematical techniques to fully leverage the limited da
ta that are available. Here we describe emph{transition path theory} (TPT), a framework originally developed for molecular simulation, and argue that it is a useful paradigm for developing mechanistic understanding of rare climate events. TPT provides a method to calculate statistical properties of the paths into the event. As an initial demonstration of the utility of TPT, we analyze a low-order model of sudden stratospheric warming (SSW), a dramatic disturbance to the polar vortex which can induce extreme cold spells at the surface in the midlatitudes. SSW events pose a major challenge for seasonal weather prediction because of their rapid, complex onset and development. Climate models struggle to capture the long-term statistics of SSW, owing to their diversity and intermittent nature. We use a stochastically forced Holton-Mass-type model with two stable states, corresponding to radiative equilibrium and a vacillating SSW-like regime. In this stochastic bistable setting, from certain probabilistic forecasts TPT facilitates estimation of dominant transition pathways and return times of transitions. These dynamical statistics are obtained by solving partial differential equations in the models phase space. With future application to more complex models, TPT and its constituent quantities promise to improve the predictability of extreme weather events, through both generation and principled evaluation of forecasts.
We present the umbrella sampling (US) technique and show that it can be used to sample extremely low probability areas of the posterior distribution that may be required in statistical analyses of data. In this approach sampling of the target likelih
ood is split into sampling of multiple biased likelihoods confined within individual umbrella windows. We show that the US algorithm is efficient and highly parallel and that it can be easily used with other existing MCMC samplers. The method allows the user to capitalize on their intuition and define umbrella windows and increase sampling accuracy along specific directions in the parameter space. Alternatively, one can define umbrella windows using an approach similar to parallel tempering. We provide a public code that implements umbrella sampling as a standalone python package. We present a number of tests illustrating the power of the US method in sampling low probability areas of the posterior and show that this ability allows a considerably more robust sampling of multi-modal distributions compared to the standard sampling methods. We also present an application of the method in a real world example of deriving cosmological constraints using the supernova type Ia data. We show that umbrella sampling can sample the posterior accurately down to the $approx 15sigma$ credible region in the $Omega_{rm m}-Omega_Lambda$ plane, while for the same computational work the affine-invariant MCMC sampling implemented in the {tt emcee} code samples the posterior reliably only to $approx 3sigma$.
