No Arabic abstract
We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {it metastable points} as a subset of the state space $G_N$ such that (i) this set is reached from any point $xin G_N$ without return to x with probability at least $b_N$, while (ii) for any two point x,y in the metastable set, the probability $T^{-1}_{x,y}$ to reach y from x without return to x is smaller than $a_N^{-1}ll b_N$. Under some additional non-degeneracy assumption, we show that in such a situation: item{(i)} To each metastable point corresponds a metastable state, whose mean exit time can be computed precisely. item{(ii)} To each metastable point corresponds one simple eigenvalue of $1-P_N$ which is essentially equal to the inverse mean exit time from this state. The corresponding eigenfunctions are close to the indicator function of the support of the metastable state. Moreover, these results imply very sharp uniform control of the deviation of the probability distribution of metastable exit times from the exponential distribution.
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available. This includes a sharp uncertainty principle relating all low-lying eigenvalues to mean times of metastable transitions, a relation between the support of eigenfunctions and the attractor of a metastable state, and sharp estimates on the convergence of probability distribution of the metastable transition times to the exponential distribution.
In this paper, we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity orbit of a normal transition kernel, such as the one of birth-death chains or reversible Markov chains. We start by identifying a set of sufficient conditions for a Markov chain to belong to the similarity orbit of a birth-death one. As by-products, we obtain a spectral representation in terms of non-self-adjoint resolutions of identity in the sense of Dunford [21] and offer a detailed analysis on the convergence rate, separation cutoff and ${rm{L}}^2$-cutoff of this class of non-reversible Markov chains. We also look into the problem of estimating the integral functionals from discrete observations for this class. In the last part of this paper, we investigate a particular similarity orbit of reversible Markov kernels, that we call the pure birth orbit, and analyze various possibly non-reversible variants of classical birth-death processes in this orbit.
We review recent results on the metastable behavior of continuous-time Markov chains derived through the characterization of Markov chains as unique solutions of martingale problems.
Dealing with finite Markov chains in discrete time, the focus often lies on convergence behavior and one tries to make different copies of the chain meet as fast as possible and then stick together. There is, however, a very peculiar kind of discrete finite Markov chain, for which two copies started in different states can be coupled to meet almost surely in finite time, yet their distributions keep a total variation distance bounded away from 0, even in the limit as time goes off to infinity. We show that the supremum of total variation distance kept in this context is $frac12$.
We introduce the space of virtual Markov chains (VMCs) as a projective limit of the spaces of all finite state space Markov chains (MCs), in the same way that the space of virtual permutations is the projective limit of the spaces of all permutations of finite sets. We introduce the notions of virtual initial distribution (VID) and a virtual transition matrix (VTM), and we show that the law of any VMC is uniquely characterized by a pair of a VID and VTM which have to satisfy a certain compatibility condition. Lastly, we study various properties of compact convex sets associated to the theory of VMCs, including that the Birkhoff-von Neumann theorem fails in the virtual setting.