Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userWell-Conditioned Methods for Ill-Conditioned Systems: Linear Regression with Semi-Random Noise
79
0
0.0
(
0
)
Added by
Kevin Tian
Publication date
2020
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Classical iterative algorithms for linear system solving and regression are brittle to the condition number of the data matrix. Even a semi-random adversary, constrained to only give additional consistent information, can arbitrarily hinder the resulting computational guarantees of existing solvers. We show how to overcome this barrier by developing a framework which takes state-of-the-art solvers and robustifies them to achieve comparable guarantees against a semi-random adversary. Given a matrix which contains an (unknown) well-conditioned submatrix, our methods obtain computational and statistical guarantees as if the entire matrix was well-conditioned. We complement our theoretical results with preliminary experimental evidence, showing that our methods are effective in practice.
rate research
Read More
We give lower bounds on the performance of two of the most popular sampling methods in practice, the Metropolis-adjusted Langevin algorithm (MALA) and multi-step Hamiltonian Monte Carlo (HMC) with a leapfrog integrator, when applied to well-conditioned distributions. Our main result is a nearly-tight lower bound of $widetilde{Omega}(kappa d)$ on the mixing time of MALA from an exponentially warm start, matching a line of algorithmic results up to logarithmic factors and answering an open question of Chewi et. al. We also show that a polynomial dependence on dimension is necessary for the relaxation time of HMC under any number of leapfrog steps, and bound the gains achievable by changing the step count. Our HMC analysis draws upon a novel connection between leapfrog integration and Chebyshev polynomials, which may be of independent interest.
We present a method for performing sampling from a Boltzmann distribution of an ill-conditioned quadratic action. This method is based on heatbath thermalization along a set of conjugate directions, generated via a conjugate-gradient procedure. The resulting scheme outperforms local updates for matrices with very high condition number, since it avoids the slowing down of modes with lower eigenvalue, and has some advantages over the global heatbath approach, compared to which it is more stable and allows for more freedom in devising case-specific optimizations.
We introduce the notion of Lyapunov exponents for random dynamical systems, conditioned to trajectories that stay within a bounded domain for asymptotically long times. This is motivated by the desire to characterize local dynamical properties in the presence of unbounded noise (when almost all trajectories are unbounded). We illustrate its use in the analysis of local bifurcations in this context. The theory of conditioned Lyapunov exponents of stochastic differential equations builds on the stochastic analysis of quasi-stationary distributions for killed processes and associated quasi-ergodic distributions. We show that conditioned Lyapunov exponents describe the local stability behaviour of trajectories that remain within a bounded domain and - in particular - that negative conditioned Lyapunov exponents imply local synchronisation. Furthermore, a conditioned dichotomy spectrum is introduced and its main characteristics are established.
A hierarchical solver is proposed for solving sparse ill-conditioned linear systems in parallel. The solver is based on a modification of the LoRaSp method, but employs a deferred-compression technique, which provably reduces the approximation error and significantly improves efficiency. Moreover, the deferred-compression technique introduces minimal overhead and does not affect parallelism. As a result, the new solver achieves linear computational complexity under mild assumptions and excellent parallel scalability. To demonstrate the performance of the new solver, we focus on applying it to solve sparse linear systems arising from ice sheet modeling. The strong anisotropic phenomena associated with the thin structure of ice sheets creates serious challenges for existing solvers. To address the anisotropy, we additionally developed a customized partitioning scheme for the solver, which captures the strong-coupling direction accurately. In general, the partitioning can be computed algebraically with existing software packages, and thus the new solver is generalizable for solving other sparse linear systems. Our results show that ice sheet problems of about 300 million degrees of freedom have been solved in just a few minutes using a thousand processors.
Product formula approximations of the time-evolution operator on quantum computers are of great interest due to their simplicity, and good scaling with system size by exploiting commutativity between Hamiltonian terms. However, product formulas exhibit poor scaling with the time $t$ and error $epsilon$ of simulation as the gate cost of a single step scales exponentially with the order $m$ of accuracy. We introduce well-conditioned multiproduct formulas, which are a linear combination of product formulas, where a single step has polynomial cost $mathcal{O}(m^2log{(m)})$ and succeeds with probability $Omega(1/operatorname{log}^2{(m)})$. Our multiproduct formulas imply a simple and generic simulation algorithm that simultaneously exploits commutativity in arbitrary systems and has a worst-case cost $mathcal{O}(tlog^{2}{(t/epsilon)})$ which is optimal up to poly-logarithmic factors. In contrast, prior Trotter and post-Trotter Hamiltonian simulation algorithms realize only one of these two desirable features. A key technical result of independent interest is our solution to a conditioning problem in previous multiproduct formulas that amplified numerical errors by $e^{Omega(m)}$ in the classical setting, and led to a vanishing success probability $e^{-Omega(m)}$ in the quantum setting.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
