Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAffine-invariant ensemble transform methods for logistic regression
187
0
0.0
(
0
)
Added by
Sebastian Reich
Publication date
2021
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
We investigate the application of ensemble transform approaches to Bayesian inference of logistic regression problems. Our approach relies on appropriate extensions of the popular ensemble Kalman filter and the feedback particle filter to the cross entropy loss function and is based on a well-established homotopy approach to Bayesian inference. The arising finite particle evolution equations as well as their mean-field limits are affine-invariant. Furthermore, the proposed methods can be implemented in a gradient-free manner in case of nonlinear logistic regression and the data can be randomly subsampled similar to mini-batching of stochastic gradient descent. We also propose a closely related SDE-based sampling method which again is affine-invariant and can easily be made gradient-free. Numerical examples demonstrate the appropriateness of the proposed methodologies.
rate research
Read More
Several numerical tools designed to overcome the challenges of smoothing in a nonlinear and non-Gaussian setting are investigated for a class of particle smoothers. The considered family of smoothers is induced by the class of linear ensemble transform filters which contains classical filters such as the stochastic ensemble Kalman filter, the ensemble square root filter and the recently introduced nonlinear ensemble transform filter. Further the ensemble transform particle smoother is introduced and particularly highlighted as it is consistent in the particle limit and does not require assumptions with respect to the family of the posterior distribution. The linear update pattern of the considered class of linear ensemble transform smoothers allows one to implement important supplementary techniques such as adaptive spread corrections, hybrid formulations, and localization in order to facilitate their application to complex estimation problems. These additional features are derived and numerically investigated for a sequence of increasingly challenging test problems.
The increasing availability of data presents an opportunity to calibrate unknown parameters which appear in complex models of phenomena in the biomedical, physical and social sciences. However, model complexity often leads to parameter-to-data maps which are expensive to evaluate and are only available through noisy approximations. This paper is concerned with the use of interacting particle systems for the solution of the resulting inverse problems for parameters. Of particular interest is the case where the available forward model evaluations are subject to rapid fluctuations, in parameter space, superimposed on the smoothly varying large scale parametric structure of interest. Multiscale analysis is used to study the behaviour of interacting particle system algorithms when such rapid fluctuations, which we refer to as noise, pollute the large scale parametric dependence of the parameter-to-data map. Ensemble Kalman methods (which are derivative-free) and Langevin-based methods (which use the derivative of the parameter-to-data map) are compared in this light. The ensemble Kalman methods are shown to behave favourably in the presence of noise in the parameter-to-data map, whereas Langevin methods are adversely affected. On the other hand, Langevin methods have the correct equilibrium distribution in the setting of noise-free forward models, whilst ensemble Kalman methods only provide an uncontrolled approximation, except in the linear case. Therefore a new class of algorithms, ensemble Gaussian process samplers, which combine the benefits of both ensemble Kalman and Langevin methods, are introduced and shown to perform favourably.
This paper explores Tadmors minimum entropy principle for the relativistic hydrodynamics (RHD) equations and incorporates this principle into the design of robust high-order discontinuous Galerkin (DG) and finite volume schemes for RHD on general meshes. The schemes are proven to preserve numerical solutions in a global invariant region constituted by all the known intrinsic constraints: minimum entropy principle, the subluminal constraint on fluid velocity, and the positivity of pressure and rest-mass density. Relativistic effects lead to some essential difficulties in the present study, which are not encountered in the non-relativistic case. Most notably, in the RHD case the specific entropy is a highly nonlinear implicit function of the conservative variables, and, moreover, there is also no explicit formula of the flux in terms of the conservative variables. In order to overcome the resulting challenges, we first propose a novel equivalent form of the invariant region, by skillfully introducing two auxiliary variables. As a notable feature, all the constraints in the novel form are explicit and linear with respect to the conservative variables. This provides a highly effective approach to theoretically analyze the invariant-region-preserving (IRP) property of schemes for RHD, without any assumption on the IRP property of the exact Riemann solver. Based on this, we prove the convexity of the invariant region and establish the generalized Lax--Friedrichs splitting properties via technical estimates, lying the foundation for our IRP analysis. It is shown that the first-order Lax--Friedrichs scheme for RHD satisfies a local minimum entropy principle and is IRP under a CFL condition. Provably IRP high-order DG and finite volume methods are developed for the RHD with the help of a simple scaling limiter. Several numerical examples demonstrate the effectiveness of the proposed schemes.
This paper gives a unified convergence analysis of additive Schwarz methods for general convex optimization problems. Resembling to the fact that additive Schwarz methods for linear problems are preconditioned Richardson methods, we prove that additive Schwarz methods for general convex optimization are in fact gradient methods. Then an abstract framework for convergence analysis of additive Schwarz methods is proposed. The proposed framework applied to linear elliptic problems agrees with the classical theory. We present applications of the proposed framework to various interesting convex optimization problems such as nonlinear elliptic problems, nonsmooth problems, and nonsharp problems.
We present a novel algorithm based on the ensemble Kalman filter to solve inverse problems involving multiscale elliptic partial differential equations. Our method is based on numerical homogenization and finite element discretization and allows to recover a highly oscillatory tensor from measurements of the multiscale solution in a computationally inexpensive manner. The properties of the approximate solution are analysed with respect to the multiscale and discretization parameters, and a convergence result is shown to hold. A reinterpretation of the solution from a Bayesian perspective is provided, and convergence of the approximate conditional posterior distribution is proved with respect to the Wasserstein distance. A numerical experiment validates our methodology, with a particular emphasis on modelling error and computational cost.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
