Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEstimation of smooth functionals in high-dimensional models: bootstrap chains and Gaussian approximation
81
0
0.0
(
0
)
Added by
Vladimir Koltchinskii
Publication date
2020
fields
Mathematical Statistics
and research's language is
English
Authors
Vladimir Koltchinskii
Ask ChatGPT about the research
No Arabic abstract
Let $X^{(n)}$ be an observation sampled from a distribution $P_{theta}^{(n)}$ with an unknown parameter $theta,$ $theta$ being a vector in a Banach space $E$ (most often, a high-dimensional space of dimension $d$). We study the problem of estimation of $f(theta)$ for a functional $f:Emapsto {mathbb R}$ of some smoothness $s>0$ based on an observation $X^{(n)}sim P_{theta}^{(n)}.$ Assuming that there exists an estimator $hat theta_n=hat theta_n(X^{(n)})$ of parameter $theta$ such that $sqrt{n}(hat theta_n-theta)$ is sufficiently close in distribution to a mean zero Gaussian random vector in $E,$ we construct a functional $g:Emapsto {mathbb R}$ such that $g(hat theta_n)$ is an asymptotically normal estimator of $f(theta)$ with $sqrt{n}$ rate provided that $s>frac{1}{1-alpha}$ and $dleq n^{alpha}$ for some $alphain (0,1).$ We also derive general upper bounds on Orlicz norm error rates for estimator $g(hat theta)$ depending on smoothness $s,$ dimension $d,$ sample size $n$ and the accuracy of normal approximation of $sqrt{n}(hat theta_n-theta).$ In particular, this approach yields asymptotically efficient estimators in some high-dimensional exponential models.
rate research
Read More
We study a problem of estimation of smooth functionals of parameter $theta $ of Gaussian shift model $$ X=theta +xi, theta in E, $$ where $E$ is a separable Banach space and $X$ is an observation of unknown vector $theta$ in Gaussian noise $xi$ with zero mean and known covariance operator $Sigma.$ In particular, we develop estimators $T(X)$ of $f(theta)$ for functionals $f:Emapsto {mathbb R}$ of Holder smoothness $s>0$ such that $$ sup_{|theta|leq 1} {mathbb E}_{theta}(T(X)-f(theta))^2 lesssim Bigl(|Sigma| vee ({mathbb E}|xi|^2)^sBigr)wedge 1, $$ where $|Sigma|$ is the operator norm of $Sigma,$ and show that this mean squared error rate is minimax optimal at least in the case of standard Gaussian shift model ($E={mathbb R}^d$ equipped with the canonical Euclidean norm, $xi =sigma Z,$ $Zsim {mathcal N}(0;I_d)$). Moreover, we determine a sharp threshold on the smoothness $s$ of functional $f$ such that, for all $s$ above the threshold, $f(theta)$ can be estimated efficiently with a mean squared error rate of the order $|Sigma|$ in a small noise setting (that is, when ${mathbb E}|xi|^2$ is small). The construction of efficient estimators is crucially based on a bootstrap chain method of bias reduction. The results could be applied to a variety of special high-dimensional and infinite-dimensional Gaussian models (for vector, matrix and functional data).
Let $X_1,dots, X_n$ be i.i.d. random variables sampled from a normal distribution $N(mu,Sigma)$ in ${mathbb R}^d$ with unknown parameter $theta=(mu,Sigma)in Theta:={mathbb R}^dtimes {mathcal C}_+^d,$ where ${mathcal C}_+^d$ is the cone of positively definite covariance operators in ${mathbb R}^d.$ Given a smooth functional $f:Theta mapsto {mathbb R}^1,$ the goal is to estimate $f(theta)$ based on $X_1,dots, X_n.$ Let $$ Theta(a;d):={mathbb R}^dtimes Bigl{Sigmain {mathcal C}_+^d: sigma(Sigma)subset [1/a, a]Bigr}, ageq 1, $$ where $sigma(Sigma)$ is the spectrum of covariance $Sigma.$ Let $hat theta:=(hat mu, hat Sigma),$ where $hat mu$ is the sample mean and $hat Sigma$ is the sample covariance, based on the observations $X_1,dots, X_n.$ For an arbitrary functional $fin C^s(Theta),$ $s=k+1+rho, kgeq 0, rhoin (0,1],$ we define a functional $f_k:Theta mapsto {mathbb R}$ such that begin{align*} & sup_{thetain Theta(a;d)}|f_k(hat theta)-f(theta)|_{L_2({mathbb P}_{theta})} lesssim_{s, beta} |f|_{C^{s}(Theta)} biggr[biggl(frac{a}{sqrt{n}} bigvee a^{beta s}biggl(sqrt{frac{d}{n}}biggr)^{s} biggr)wedge 1biggr], end{align*} where $beta =1$ for $k=0$ and $beta>s-1$ is arbitrary for $kgeq 1.$ This error rate is minimax optimal and similar bounds hold for more general loss functions. If $d=d_nleq n^{alpha}$ for some $alphain (0,1)$ and $sgeq frac{1}{1-alpha},$ the rate becomes $O(n^{-1/2}).$ Moreover, for $s>frac{1}{1-alpha},$ the estimators $f_k(hat theta)$ is shown to be asymptotically efficient. The crucial part of the construction of estimator $f_k(hat theta)$ is a bias reduction method studied in the paper for more general statistical models than normal.
Let $X$ be a centered Gaussian random variable in a separable Hilbert space ${mathbb H}$ with covariance operator $Sigma.$ We study a problem of estimation of a smooth functional of $Sigma$ based on a sample $X_1,dots ,X_n$ of $n$ independent observations of $X.$ More specifically, we are interested in functionals of the form $langle f(Sigma), Brangle,$ where $f:{mathbb R}mapsto {mathbb R}$ is a smooth function and $B$ is a nuclear operator in ${mathbb H}.$ We prove concentration and normal approximation bounds for plug-in estimator $langle f(hat Sigma),Brangle,$ $hat Sigma:=n^{-1}sum_{j=1}^n X_jotimes X_j$ being the sample covariance based on $X_1,dots, X_n.$ These bounds show that $langle f(hat Sigma),Brangle$ is an asymptotically normal estimator of its expectation ${mathbb E}_{Sigma} langle f(hat Sigma),Brangle$ (rather than of parameter of interest $langle f(Sigma),Brangle$) with a parametric convergence rate $O(n^{-1/2})$ provided that the effective rank ${bf r}(Sigma):= frac{{bf tr}(Sigma)}{|Sigma|}$ (${rm tr}(Sigma)$ being the trace and $|Sigma|$ being the operator norm of $Sigma$) satisfies the assumption ${bf r}(Sigma)=o(n).$ At the same time, we show that the bias of this estimator is typically as large as $frac{{bf r}(Sigma)}{n}$ (which is larger than $n^{-1/2}$ if ${bf r}(Sigma)geq n^{1/2}$). In the case when ${mathbb H}$ is finite-dimensional space of dimension $d=o(n),$ we develop a method of bias reduction and construct an estimator $langle h(hat Sigma),Brangle$ of $langle f(Sigma),Brangle$ that is asymptotically normal with convergence rate $O(n^{-1/2}).$ Moreover, we study asymptotic properties of the risk of this estimator and prove minimax lower bounds for arbitrary estimators showing the asymptotic efficiency of $langle h(hat Sigma),Brangle$ in a semi-parametric sense.
Let $(Y,(X_i)_{iinmathcal{I}})$ be a zero mean Gaussian vector and $V$ be a subset of $mathcal{I}$. Suppose we are given $n$ i.i.d. replications of the vector $(Y,X)$. We propose a new test for testing that $Y$ is independent of $(X_i)_{iin mathcal{I}backslash V}$ conditionally to $(X_i)_{iin V}$ against the general alternative that it is not. This procedure does not depend on any prior information on the covariance of $X$ or the variance of $Y$ and applies in a high-dimensional setting. It straightforwardly extends to test the neighbourhood of a Gaussian graphical model. The procedure is based on a model of Gaussian regression with random Gaussian covariates. We give non asymptotic properties of the test and we prove that it is rate optimal (up to a possible $log(n)$ factor) over various classes of alternatives under some additional assumptions. Besides, it allows us to derive non asymptotic minimax rates of testing in this setting. Finally, we carry out a simulation study in order to evaluate the performance of our procedure.
In this paper, we address the estimation of the sensitivity indices called Shapley eects. These sensitivity indices enable to handle dependent input variables. The Shapley eects are generally dicult to estimate, but they are easily computable in the Gaussian linear framework. The aim of this work is to use the values of the Shapley eects in an approximated Gaussian linear framework as estimators of the true Shapley eects corresponding to a non-linear model. First, we assume that the input variables are Gaussian with small variances. We provide rates of convergence of the estimated Shapley eects to the true Shapley eects. Then, we focus on the case where the inputs are given by an non-Gaussian empirical mean. We prove that, under some mild assumptions, when the number of terms in the empirical mean increases, the dierence between the true Shapley eects and the estimated Shapley eects given by the Gaussian linear approximation converges to 0. Our theoretical results are supported by numerical studies, showing that the Gaussian linear approximation is accurate and enables to decrease the computational time signicantly.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
