Let $Ainmathbb{R}^{mtimes n}$ be a matrix of rank $r$ with singular value decomposition (SVD) $A=sum_{k=1}^rsigma_k (u_kotimes v_k),$ where ${sigma_k, k=1,ldots,r}$ are singular values of $A$ (arranged in a non-increasing order) and $u_kin {mathbb R}^m, v_kin {mathbb R}^n, k=1,ldots, r$ are the corresponding left and right orthonormal singular vectors. Let $tilde{A}=A+X$ be a noisy observation of $A,$ where $Xinmathbb{R}^{mtimes n}$ is a random matrix with i.i.d. Gaussian entries, $X_{ij}simmathcal{N}(0,tau^2),$ and consider its SVD $tilde{A}=sum_{k=1}^{mwedge n}tilde{sigma}_k(tilde{u}_kotimestilde{v}_k)$ with singular values $tilde{sigma}_1geqldotsgeqtilde{sigma}_{mwedge n}$ and singular vectors $tilde{u}_k,tilde{v}_k,k=1,ldots, mwedge n.$ The goal of this paper is to develop sharp concentration bounds for linear forms $langle tilde u_k,xrangle, xin {mathbb R}^m$ and $langle tilde v_k,yrangle, yin {mathbb R}^n$ of the perturbed (empirical) singular vectors in the case when the singular values of $A$ are distinct and, more generally, concentration bounds for bilinear forms of projection operators associated with SVD. In particular, the results imply upper bounds of the order $Obiggl(sqrt{frac{log(m+n)}{mvee n}}biggr)$ (holding with a high probability) on $$max_{1leq ileq m}big|big<tilde{u}_k-sqrt{1+b_k}u_k,e_i^mbig>big| {rm and} max_{1leq jleq n}big|big<tilde{v}_k-sqrt{1+b_k}v_k,e_j^nbig>big|,$$ where $b_k$ are properly chosen constants characterizing the bias of empirical singular vectors $tilde u_k, tilde v_k$ and ${e_i^m,i=1,ldots,m}, {e_j^n,j=1,ldots,n}$ are the canonical bases of $mathbb{R}^m, {mathbb R}^n,$ respectively.
Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the lower tail of such a matrix, and prove that it is subgaussian under a simple fourth moment assumption on the one-dimensional marginals of the random vectors. A similar result holds for more general sums of random positive semidefinite matrices, and the (relatively simple) proof uses a variant of the so-called PAC-Bayesian method for bounding empirical processes. We give two applications of the main result. In the first one we obtain a new finite-sample bound for ordinary least squares estimator in linear regression with random design. Our result is model-free, requires fairly weak moment assumptions and is almost optimal. Our second application is to bounding restricted eigenvalue constants of certain random ensembles with heavy tails. These constants are important in the analysis of problems in Compressed Sensing and High Dimensional Statistics, where one recovers a sparse vector from a small umber of linear measurements. Our result implies that heavy tails still allow for the fast recovery rates found in efficient methods such as the LASSO and the Dantzig selector. Along the way we strengthen, with a fairly short argument, a recent result of Rudelson and Zhou on the restricted eigenvalue property.
In this article, we obtain an upper bound for the number of integral solutions, of given height, of system of two quadratic forms in five variables. Our bound is an improvement over the bound given by Henryk Iwaniec and Ritabrata Munshi in cite{H-R}.