Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userShrinkage Estimation of the Power Spectrum Covariance Matrix
115
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
We seek to improve estimates of the power spectrum covariance matrix from a limited number of simulations by employing a novel statistical technique known as shrinkage estimation. The shrinkage technique optimally combines an empirical estimate of the covariance with a model (the target) to minimize the total mean squared error compared to the true underlying covariance. We test this technique on N-body simulations and evaluate its performance by estimating cosmological parameters. Using a simple diagonal target, we show that the shrinkage estimator significantly outperforms both the empirical covariance and the target individually when using a small number of simulations. We find that reducing noise in the covariance estimate is essential for properly estimating the values of cosmological parameters as well as their confidence intervals. We extend our method to the jackknife covariance estimator and again find significant improvement, though simulations give better results. Even for thousands of simulations we still find evidence that our method improves estimation of the covariance matrix. Because our method is simple, requires negligible additional numerical effort, and produces superior results, we always advocate shrinkage estimation for the covariance of the power spectrum and other large-scale structure measurements when purely theoretical modeling of the covariance is insufficient.
rate research
Read More
We show how to estimate the covariance of the power spectrum of a statistically homogeneous and isotropic density field from a single periodic simulation, by applying a set of weightings to the density field, and by measuring the scatter in power spectra between different weightings. We recommend a specific set of 52 weightings containing only combinations of fundamental modes, constructed to yield a minimum variance estimate of the covariance of power. Numerical tests reveal that at nonlinear scales the variance of power estimated by the weightings method substantially exceeds that estimated from a simple ensemble method. We argue that the discrepancy is caused by beat-coupling, in which products of closely spaced Fourier modes couple by nonlinear gravitational growth to the beat mode between them. Beat-coupling appears whenever nonlinear power is measured from Fourier modes with a finite spread of wavevector, and is therefore present in the weightings method but not the ensemble method. Beat-coupling inevitably affects real galaxy surveys, whose Fourier modes have finite width. Surprisingly, the beat-coupling contribution dominates the covariance of power at nonlinear scales, so that, counter-intuitively, it is expected that the covariance of nonlinear power in galaxy surveys is dominated not by small scale structure, but rather by beat-coupling to the largest scales of the survey.
This paper considers the regularized estimation of covariance matrices (CM) of high-dimensional (compound) Gaussian data for minimum variance distortionless response (MVDR) beamforming. Linear shrinkage is applied to improve the accuracy and condition number of the CM estimate for low-sample-support cases. We focus on data-driven techniques that automatically choose the linear shrinkage factors for shrinkage sample covariance matrix ($text{S}^2$CM) and shrinkage Tylers estimator (STE) by exploiting cross validation (CV). We propose leave-one-out cross-validation (LOOCV) choices for the shrinkage factors to optimize the beamforming performance, referred to as $text{S}^2$CM-CV and STE-CV. The (weighted) out-of-sample output power of the beamfomer is chosen as a proxy of the beamformer performance and concise expressions of the LOOCV cost function are derived to allow fast optimization. For the large system regime, asymptotic approximations of the LOOCV cost functions are derived, yielding the $text{S}^2$CM-AE and STE-AE. In general, the proposed algorithms are able to achieve near-oracle performance in choosing the linear shrinkage factors for MVDR beamforming. Simulation results are provided for validating the proposed methods.
We use 5000 cosmological N-body simulations of 1(Gpc/h)^3 box for the concordance LCDM model in order to study the sampling variances of nonlinear matter power spectrum. We show that the non-Gaussian errors can be important even on large length scales relevant for baryon acoustic oscillations (BAO). Our findings are (1) the non-Gaussian errors degrade the cumulative signal-to-noise ratios (S/N) for the power spectrum amplitude by up to a factor of 2 and 4 for redshifts z=1 and 0, respectively. (2) There is little information on the power spectrum amplitudes in the quasi-nonlinear regime, confirming the previous results. (3) The distribution of power spectrum estimators at BAO scales, among the realizations, is well approximated by a Gaussian distribution with variance that is given by the diagonal covariance component. (4) For the redshift-space power spectrum, the degradation in S/N by non-Gaussian errors is mitigated due to nonlinear redshift distortions. (5) For an actual galaxy survey, the additional shot noise contamination compromises the cosmological information inherent in the galaxy power spectrum, but also mitigates the impact of non-Gaussian errors. The S/N is degraded by up to 30% for a WFMOS-type survey. (6) The finite survey volume causes additional non-Gaussian errors via the correlations of long-wavelength fluctuations with the fluctuations we want to measure, further degrading the S/N values by about 30% even at high redshift z=3.
Portfolio managers faced with limited sample sizes must use factor models to estimate the covariance matrix of a high-dimensional returns vector. For the simplest one-factor market model, success rests on the quality of the estimated leading eigenvector beta. When only the returns themselves are observed, the practitioner has available the PCA estimate equal to the leading eigenvector of the sample covariance matrix. This estimator performs poorly in various ways. To address this problem in the high-dimension, limited sample size asymptotic regime and in the context of estimating the minimum variance portfolio, Goldberg, Papanicolau, and Shkolnik developed a shrinkage method (the GPS estimator) that improves the PCA estimator of beta by shrinking it toward a constant target unit vector. In this paper we continue their work to develop a more general framework of shrinkage targets that allows the practitioner to make use of further information to improve the estimator. Examples include sector separation of stock betas, and recent information from prior estimates. We prove some precise statements and illustrate the resulting improvements over the GPS estimator with some numerical experiments.
Consider estimating the n by p matrix of means of an n by p matrix of independent normally distributed observations with constant variance, where the performance of an estimator is judged using a p by p matrix quadratic error loss function. A matrix version of the James-Stein estimator is proposed, depending on a tuning constant. It is shown to dominate the usual maximum likelihood estimator for some choices of of the tuning constant when n is greater than or equal to 3. This result also extends to other shrinkage estimators and settings.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
