Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA reinforcement learning approach to rare trajectory sampling
194
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Very often when studying non-equilibrium systems one is interested in analysing dynamical behaviour that occurs with very low probability, so called rare events. In practice, since rare events are by definition atypical, they are often difficult to access in a statistically significant way. What are required are strategies to make rare events typical so that they can be generated on demand. Here we present such a general approach to adaptively construct a dynamics that efficiently samples atypical events. We do so by exploiting the methods of reinforcement learning (RL), which refers to the set of machine learning techniques aimed at finding the optimal behaviour to maximise a reward associated with the dynamics. We consider the general perspective of dynamical trajectory ensembles, whereby rare events are described in terms of ensemble reweighting. By minimising the distance between a reweighted ensemble and that of a suitably parametrised controlled dynamics we arrive at a set of methods similar to those of RL to numerically approximate the optimal dynamics that realises the rare behaviour of interest. As simple illustrations we consider in detail the problem of excursions of a random walker, for the case of rare events with a finite time horizon; and the problem of a studying current statistics of a particle hopping in a ring geometry, for the case of an infinite time horizon. We discuss natural extensions of the ideas presented here, including to continuous-time Markov systems, first passage time problems and non-Markovian dynamics.
rate research
Read More
In stochastic systems, numerically sampling the relevant trajectories for the estimation of the large deviation statistics of time-extensive observables requires overcoming their exponential (in space and time) scarcity. The optimal way to access these rare events is by means of an auxiliary dynamics obtained from the original one through the so-called ``generalised Doob transformation. While this optimal dynamics is guaranteed to exist its use is often impractical, as to define it requires the often impossible task of diagonalising a (tilted) dynamical generator. While approximate schemes have been devised to overcome this issue they are difficult to automate as they tend to require knowledge of the systems under study. Here we address this problem from the perspective of deep learning. We devise an iterative semi-supervised learning scheme which converges to the optimal or Doob dynamics with the clear advantage of requiring no prior knowledge of the system. We test our method in a paradigmatic statistical mechanics model with non-trivial dynamical fluctuations, the fully packed classical dimer model on the square lattice, showing that it compares favourably with more traditional approaches. We discuss broader implications of our results for the study of rare dynamical trajectories.
We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, like those relevant in the study of reactive events, as well as trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, like those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.
We study the time until first occurrence, the first-passage time, of rare density fluctuations in diffusive systems. We approach the problem using a model consisting of many independent random walkers on a lattice. The existence of spatial correlations makes this problem analytically intractable. However, for a mean-field approximation in which the walkers can jump anywhere in the system, we obtain a simple asymptotic form for the mean first-passage time to have a given number k of particles at a distinguished site. We show numerically, and argue heuristically, that for large enough k, the mean-field results give a good approximation for first-passage times for systems with nearest-neighbour dynamics, especially for two and higher spatial dimensions. Finally, we show how the results change when density fluctuations anywhere in the system, rather than at a specific distinguished site, are considered.
The inverse problem for a disordered system involves determining the interparticle interaction parameters consistent with a given set of experimental data. Recently, Rutledge has shown (Phys. Rev. E63, 021111 (2001)) that such problems can be generally expressed in terms of a grand canonical ensemble of polydisperse particles. Within this framework, one identifies a polydisperse attribute (`pseudo-species) $sigma$ corresponding to some appropriate generalized coordinate of the system to hand. Associated with this attribute is a composition distribution $barrho(sigma)$ measuring the number of particles of each species. Its form is controlled by a conjugate chemical potential distribution $mu(sigma)$ which plays the role of the requisite interparticle interaction potential. Simulation approaches to the inverse problem involve determining the form of $mu(sigma)$ for which $barrho(sigma)$ matches the available experimental data. The difficulty in doing so is that $mu(sigma)$ is (in general) an unknown {em functional} of $barrho(sigma)$ and must therefore be found by iteration. At high particle densities and for high degrees of polydispersity, strong cross coupling between $mu(sigma)$ and $barrho(sigma)$ renders this process computationally problematic and laborious. Here we describe an efficient and robust {em non-equilibrium} simulation scheme for finding the equilibrium form of $mu[barrho(sigma)]$. The utility of the method is demonstrated by calculating the chemical potential distribution conjugate to a specific log-normal distribution of particle sizes in a polydisperse fluid.
A Boltzmann machine is a stochastic neural network that has been extensively used in the layers of deep architectures for modern machine learning applications. In this paper, we develop a Boltzmann machine that is capable of modelling thermodynamic observables for physical systems in thermal equilibrium. Through unsupervised learning, we train the Boltzmann machine on data sets constructed with spin configurations importance-sampled from the partition function of an Ising Hamiltonian at different temperatures using Monte Carlo (MC) methods. The trained Boltzmann machine is then used to generate spin states, for which we compare thermodynamic observables to those computed by direct MC sampling. We demonstrate that the Boltzmann machine can faithfully reproduce the observables of the physical system. Further, we observe that the number of neurons required to obtain accurate results increases as the system is brought close to criticality.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
