Sharp Asymptotic Estimates for Expectations, Probabilities, and Mean First Passage Times in Stochastic Systems with Small Noise

Freidlin-Wentzell theory of large deviations can be used to compute the likelihood of extreme or rare events in stochastic dynamical systems via the solution of an optimization problem. The approach gives exponential estimates that often need to be refined via calculation of a prefactor. Here it is shown how to perform these computations in practice. Specifically, sharp asymptotic estimates are derived for expectations, probabilities, and mean first passage times in a form that is geared towards numerical purposes: they require solving well-posed matrix Riccati equations involving the minimizer of the Freidlin-Wentzell action as input, either forward or backward in time with appropriate initial or final conditions tailored to the estimate at hand. The usefulness of our approach is illustrated on several examples. In particular, invariant measure probabilities and mean first passage times are calculated in models involving stochastic partial differential equations of reaction-advection-diffusion type.

Active Importance Sampling for Variational Objectives Dominated by Rare Events: Consequences for Optimization and Generalization

Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with dimensionality. As a result, many high-dimensional sampling and approximation problems once thought intractable are being revisited through the lens of machine learning. While the promise of unparalleled accuracy may suggest a renaissance for applications that require parameterizing representations of complex systems, in many applications gathering sufficient data to develop such a representation remains a significant challenge. Here we introduce an approach that combines rare events sampling techniques with neural network optimization to optimize objective functions that are dominated by rare events. We show that importance sampling reduces the asymptotic variance of the solution to a learning problem, suggesting benefits for generalization. We study our algorithm in the context of learning dynamical transition pathways between two states of a system, a problem with applications in statistical physics and implications in machine learning theory. Our numerical experiments demonstrate that we can successfully learn even with the compounding difficulties of high-dimension and rare data.

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the over-parametrized regime where the layer width $m$ is larger than the input dimension $d$. We consider a teacher-student scenario where the teacher has the same structure as the student with a hidden layer of smaller width $m^*le m$. We describe how the empirical loss landscape is affected by the number $n$ of data samples and the width $m^*$ of the teacher network. In particular we determine how the probability that there be no spurious minima on the empirical loss depends on $n$, $d$, and $m^*$, thereby establishing conditions under which the neural network can in principle recover the teacher. We also show that under the same conditions gradient descent dynamics on the empirical loss converges and leads to small generalization error, i.e. it enables recovery in practice. Finally we characterize the time-convergence rate of gradient descent in the limit of a large number of samples. These results are confirmed by numerical experiments.

Infinite Switch Simulated Tempering in Force (FISST)

Many proteins in cells are capable of sensing and responding to piconewton scale forces, a regime in which conformational changes are small but significant for biological processes. In order to efficiently and effectively sample the response of these proteins to small forces, enhanced sampling techniques will be required. In this work, we derive, implement, and evaluate an efficient method to simultaneously sample the result of applying any constant pulling force within a specified range to a molecular system of interest. We start from Simulated Tempering in Force, whereby force is applied as a linear bias on a collective variable to the systems Hamiltonian, and the coefficient is taken as a continuous auxiliary degree of freedom. We derive a formula for an average collective-variable-dependent force, which depends on a set of weights, learned on-the-fly throughout a simulation, that reflect the limit where force varies infinitely quickly. These weights can then be used to retroactively compute averages of any observable at any force within the specified range. This technique is based on recent work deriving similar equations for Infinite Switch Simulated Tempering in Temperature, that showed the infinite switch limit is the most efficient for sampling. Here, we demonstrate that our method accurately and simultaneously samples molecular systems at all forces within a user defined force range, and show how it can serve as an enhanced sampling tool for cases where the pulling direction destabilizes states of low free-energy at zero-force. This method is implemented in, and will be freely-distributed with, the PLUMED open-source sampling library, and hence can be readily applied to problems using a wide range of molecular dynamics software packages.

Longtime convergence of the Temperature-Accelerated Molecular Dynamics Method

The equations of the temperature-accelerated molecular dynamics (TAMD) method for the calculations of free energies and partition functions are analyzed. Specifically, the exponential convergence of the law of these stochastic processes is established, with a convergence rate close to the one of the limiting, effective dynamics at higher temperature obtained with infinite acceleration. It is also shown that the invariant measures of TAMD are close to a known reference measure, with an error that can be quantified precisely. Finally, a Central Limit Theorem is proven, which allows the estimation of errors on properties calculated by ergodic time averages. These results not only demonstrate the usefulness and validity range of the TAMD equations, but they also permit in principle to adjust the parameter in these equations to optimize their efficiency.

Averaged equation for energy diffusion on a graph reveals bifurcation diagram and thermally assisted reversal times in spin-torque driven nanomagnets

Driving nanomagnets by spin-polarized currents offers exciting prospects in magnetoelectronics, but the response of the magnets to such currents remains poorly understood. We show that an averaged equation describing the diffusion of energy on a graph captures the low-damping dynamics of these systems. From this equation we obtain the bifurcation diagram of the magnets, including the critical currents to induce stable precessional states and magnetization switching, as well as the mean times of thermally assisted magnetization reversal in situations where the standard reaction rate theory of Kramers is no longer valid. These results match experimental observations and give a theoretical basis for a Neel-Brown-type formula with an effective energy barrier for the reversal times.