Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userHierarchies, entropy, and quantitative propagation of chaos for mean field diffusions
64
0
0.0
(
0
)
Authors
Daniel Lacker
Ask ChatGPT about the research
No Arabic abstract
This paper develops a non-asymptotic, local approach to quantitative propagation of chaos for a wide class of mean field diffusive dynamics. For a system of $n$ interacting particles, the relative entropy between the marginal law of $k$ particles and its limiting product measure is shown to be $O((k/n)^2)$ at each time, as long as the same is true at time zero. A simple Gaussian example shows that this rate is optimal. The main assumption is that the limiting measure obeys a certain functional inequality, which is shown to encompass many potentially irregular but not too singular finite-range interactions, as well as some infinite-range interactions. This unifies the previously disparate cases of Lipschitz versus bounded measurable interactions, improving the best prior bounds of $O(k/n)$ which were deduced from global estimates involving all $n$ particles. We also cover a class of models for which qualitative propagation of chaos and even well-posedness of the McKean-Vlasov equation were previously unknown. At the center of a new approach is a differential inequality, derived from a form of the BBGKY hierarchy, which bounds the $k$-particle entropy in terms of the $(k+1)$-particle entropy.
rate research
Read More
We address propagation of chaos for large systems of rough differential equations associated with random rough differential equations of mean field type $$ dX_t = V(X_t,mathcal{L}(X_t))dt + F(X_t,mathcal{L}(X_t))dW_t $$ where $W$ is a random rough path and $mathcal{L}(X_t)$ is the law of $X_t$. We prove propagation of chaos, and provide also an explicit optimal convergence rate. The analysis is based upon the tools we developed in our companion paper [1] for solving mean field rough differential equations and in particular upon a corresponding version of the It^o-Lyons continuity theorem. The rate of convergence is obtained by a coupling argument developed first by Sznitman for particle systems with Brownian inputs.
Many Gibbs measures with mean field interactions are known to be chaotic, in the sense that any collection of $k$ particles in the $n$-particle system are asymptotically independent, as $ntoinfty$ with $k$ fixed or perhaps $k=o(n)$. This paper quantifies this notion for a class of continuous Gibbs measures on Euclidean space with pairwise interactions, with main examples being systems governed by convex interactions and uniformly convex confinement potentials. The distance between the marginal law of $k$ particles and its limiting product measure is shown to be $O((k/n)^{c wedge 2})$, with $c$ proportional to the squared temperature. In the high temperature case, this improves upon prior results based on subadditivity of entropy, which yield $O(k/n)$ at best. The bound $O((k/n)^2)$ cannot be improved, as a Gaussian example demonstrates. The results are non-asymptotic, and distance is quantified via relative Fisher information, relative entropy, or the squared quadratic Wasserstein metric. The method relies on an a priori functional inequality for the limiting measure, used to derive an estimate for the $k$-particle distance in terms of the $(k+1)$-particle distance.
We consider $mathbb{R}^d$-valued diffusion processes of type begin{align*} dX_t = b(X_t)dt, +, dB_t. end{align*} Assuming a geometric drift condition, we establish contractions of the transitions kernels in Kantorovich ($L^1$ Wasserstein) distances with explicit constants. Our results are in the spirit of Hairer and Mattinglys extension of Harris Theorem. In particular, they do not rely on a small set condition. Instead we combine Lyapunov functions with reflection coupling and concave distance functions. We retrieve constants that are explicit in parameters which can be computed with little effort from one-sided Lipschitz conditions for the drift coefficient and the growth of a chosen Lyapunov function. Consequences include exponential convergence in weighted total variation norms, gradient bounds, bounds for ergodic averages, and Kantorovich contractions for nonlinear McKean-Vlasov diffusions in the case of sufficiently weak but not necessarily bounded nonlinearities. We also establish quantitative bounds for sub-geometric ergodicity assuming a sub-geometric drift condition.
In this paper, we investigate the limiting behavior of a continuous-time counterpart of the Stochastic Gradient Descent (SGD) algorithm applied to two-layer overparameterized neural networks, as the number or neurons (ie, the size of the hidden layer) $N to +infty$. Following a probabilistic approach, we show propagation of chaos for the particle system defined by this continuous-time dynamics under different scenarios, indicating that the statistical interaction between the particles asymptotically vanishes. In particular, we establish quantitative convergence with respect to $N$ of any particle to a solution of a mean-field McKean-Vlasov equation in the metric space endowed with the Wasserstein distance. In comparison to previous works on the subject, we consider settings in which the sequence of stepsizes in SGD can potentially depend on the number of neurons and the iterations. We then identify two regimes under which different mean-field limits are obtained, one of them corresponding to an implicitly regularized version of the minimization problem at hand. We perform various experiments on real datasets to validate our theoretical results, assessing the existence of these two regimes on classification problems and illustrating our convergence results.
We consider Kacs 1D N-particle system coupled to an ideal thermostat at temperature T, introduced by Bonetto, Loss, and Vaidyanathan in 2014. We obtain a propagation of chaos result for this system, with explicit and uniform-in-time rates of order N^(-1/3) in the 2-Wasserstein metric. We also show well-posedness and equilibration for the limit kinetic equation in the space of probability measures. The proofs use a coupling argument previously introduced by Cortez and Fontbona in 2016.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
