Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAn Online Projection Estimator for Nonparametric Regression in Reproducing Kernel Hilbert Spaces
232
0
0.0
(
0
)
Added by
Tianyu Zhang
Publication date
2021
fields
Mathematical Statistics
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
The goal of nonparametric regression is to recover an underlying regression function from noisy observations, under the assumption that the regression function belongs to a pre-specified infinite dimensional function space. In the online setting, when the observations come in a stream, it is generally computationally infeasible to refit the whole model repeatedly. There are as of yet no methods that are both computationally efficient and statistically rate-optimal. In this paper, we propose an estimator for online nonparametric regression. Notably, our estimator is an empirical risk minimizer (ERM) in a deterministic linear space, which is quite different from existing methods using random features and functional stochastic gradient. Our theoretical analysis shows that this estimator obtains rate-optimal generalization error when the regression function is known to live in a reproducing kernel Hilbert space. We also show, theoretically and empirically, that the computational expense of our estimator is much lower than other rate-optimal estimators proposed for this online setting.
rate research
Read More
Let $G$ be a locally compact abelian group with a Haar measure, and $Y$ be a measure space. Suppose that $H$ is a reproducing kernel Hilbert space of functions on $Gtimes Y$, such that $H$ is naturally embedded into $L^2(Gtimes Y)$ and is invariant under the translations associated with the elements of $G$. Under some additional technical assumptions, we study the W*-algebra $mathcal{V}$ of translation-invariant bounded linear operators acting on $H$. First, we decompose $mathcal{V}$ into the direct integral of the W*-algebras of bounded operators acting on the reproducing kernel Hilbert spaces $widehat{H}_xi$, $xiinwidehat{G}$, generated by the Fourier transform of the reproducing kernel. Second, we give a constructive criterion for the commutativity of $mathcal{V}$. Third, in the commutative case, we construct a unitary operator that simultaneously diagonalizes all operators belonging to $mathcal{V}$, i.e., converts them into some multiplication operators. Our scheme generalizes many examples previously studied by Nikolai Vasilevski and other authors.
We prove two new equivalences of the Feichtinger conjecture that involve reproducing kernel Hilbert spaces. We prove that if for every Hilbert space, contractively contained in the Hardy space, each Bessel sequence of normalized kernel functions can be partitioned into finitely many Riesz basic sequences, then a general bounded Bessel sequence in an arbitrary Hilbert space can be partitioned into finitely many Riesz basic sequences. In addition, we examine some of these spaces and prove that for these spaces bounded Bessel sequences of normalized kernel functions are finite unions of Riesz basic sequences.
The Gaussian kernel plays a central role in machine learning, uncertainty quantification and scattered data approximation, but has received relatively little attention from a numerical analysis standpoint. The basic problem of finding an algorithm for efficient numerical integration of functions reproduced by Gaussian kernels has not been fully solved. In this article we construct two classes of algorithms that use $N$ evaluations to integrate $d$-variate functions reproduced by Gaussian kernels and prove the exponential or super-algebraic decay of their worst-case errors. In contrast to earlier work, no constraints are placed on the length-scale parameter of the Gaussian kernel. The first class of algorithms is obtained via an appropriate scaling of the classical Gauss-Hermite rules. For these algorithms we derive lower and upper bounds on the worst-case error of the forms $exp(-c_1 N^{1/d}) N^{1/(4d)}$ and $exp(-c_2 N^{1/d}) N^{-1/(4d)}$, respectively, for positive constants $c_1 > c_2$. The second class of algorithms we construct is more flexible and uses worst-case optimal weights for points that may be taken as a nested sequence. For these algorithms we derive upper bounds of the form $exp(-c_3 N^{1/(2d)})$ for a positive constant $c_3$.
In this paper, we introduce the notion of reproducing kernel Hilbert spaces for graphs and the Gram matrices associated with them. Our aim is to investigate the Gram matrices of reproducing kernel Hilbert spaces. We provide several bounds on the entries of the Gram matrices of reproducing kernel Hilbert spaces and characterize the graphs which attain our bounds.
The geometry of spaces with indefinite inner product, known also as Krein spaces, is a basic tool for developing Operator Theory therein. In the present paper we establish a link between this geometry and the algebraic theory of *-semigroups. It goes via the positive definite functions and related to them reproducing kernel Hilbert spaces. Our concern is in describing properties of elements of the semigroup which determine shift operators which serve as Pontryagin fundamental symmetries
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
