Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAdjoint-based exact Hessian computation
172
0
0.0
(
0
)
Added by
Yuto Miyatake
Publication date
2019
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
We consider a scalar function depending on a numerical solution of an initial value problem, and its second-derivative (Hessian) matrix for the initial value. The need to extract the information of the Hessian or to solve a linear system having the Hessian as a coefficient matrix arises in many research fields such as optimization, Bayesian estimation, and uncertainty quantification. From the perspective of memory efficiency, these tasks often employ a Krylov subspace method that does not need to hold the Hessian matrix explicitly and only requires computing the multiplication of the Hessian and a given vector. One of the ways to obtain an approximation of such Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant even if the numerical integration to the second-order adjoint system is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly and efficiently. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge--Kutta methods play an essential role.
rate research
Read More
The stable operation of gas networks is an important optimization target. While for this task commonly finite volume methods are used, we introduce a new finite difference approach. With a summation by part formulation for the spatial discretization, we get well-defined fluxes between the pipes. This allows a simple and explicit formulation of the coupling conditions at the node. From that, we derive the adjoint equations for the network simply and transparently. The resulting direct and adjoint equations are numerically efficient and easy to implement. The approach is demonstrated by the optimization of two sample gas networks.
We introduce a fast algorithm for entry-wise evaluation of the Gauss-Newton Hessian (GNH) matrix for the fully-connected feed-forward neural network. The algorithm has a precomputation step and a sampling step. While it generally requires $O(Nn)$ work to compute an entry (and the entire column) in the GNH matrix for a neural network with $N$ parameters and $n$ data points, our fast sampling algorithm reduces the cost to $O(n+d/epsilon^2)$ work, where $d$ is the output dimension of the network and $epsilon$ is a prescribed accuracy (independent of $N$). One application of our algorithm is constructing the hierarchical-matrix (H-matrix) approximation of the GNH matrix for solving linear systems and eigenvalue problems. It generally requires $O(N^2)$ memory and $O(N^3)$ work to store and factorize the GNH matrix, respectively. The H-matrix approximation requires only $O(N r_o)$ memory footprint and $O(N r_o^2)$ work to be factorized, where $r_o ll N$ is the maximum rank of off-diagonal blocks in the GNH matrix. We demonstrate the performance of our fast algorithm and the H-matrix approximation on classification and autoencoder neural networks.
This study computes the gradient of a function of numerical solutions of ordinary differential equations (ODEs) with respect to the initial condition. The adjoint method computes the gradient approximately by solving the corresponding adjoint system numerically. In this context, Sanz-Serna [SIAM Rev., 58 (2016), pp. 3--33] showed that when the initial value problem is solved by a Runge--Kutta (RK) method, the gradient can be exactly computed by applying an appropriate RK method to the adjoint system. Focusing on the case where the initial value problem is solved by a partitioned RK (PRK) method, this paper presents a numerical method, which can be seen as a generalization of PRK methods, for the adjoint system that gives the exact gradient.
Recently developed concept of dissipative measure-valued solution for compressible flows is a suitable tool to describe oscillations and singularities possibly developed in solutions of multidimensional Euler equations. In this paper we study the convergence of the first-order finite volume method based on the exact Riemann solver for the complete compressible Euler equations. Specifically, we derive entropy inequality and prove the consistency of numerical method. Passing to the limit, we show the weak and strong convergence of numerical solutions and identify their limit. The numerical results presented for the spiral, Kelvin-Helmholtz and the Richtmyer-Meshkov problem are consistent with our theoretical analysis.
We construct several smooth finite element spaces defined on three--dimensional Worsey--Farin splits. In particular, we construct $C^1$, $H^1(curl)$, and $H^1$-conforming finite element spaces and show the discrete spaces satisfy local exactness properties. A feature of the spaces is their low polynomial degree and lack of extrinsic supersmoothness at sub-simplices of the mesh. In the lowest order case, the last two spaces in the sequence consist of piecewise linear and piecewise constant spaces, and are suitable for the discretization of the (Navier-)Stokes equation.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
