اشترك بالحزمة الذهبية واحصل على وصول غير محدود شمرا أكاديميا
تسجيل مستخدم جديدEstimation for High-Dimensional Multi-Layer Generalized Linear Model -- Part I: The Exact MMSE Estimator
55
0
0.0
(
0
)
اسأل ChatGPT حول البحث
ﻻ يوجد ملخص باللغة العربية
This two-part work considers the minimum means square error (MMSE) estimation problem for a high dimensional multi-layer generalized linear model (ML-GLM), which resembles a feed-forward fully connected deep learning network in that each of its layer mixes up the random input with a known weighting matrix and activates the results via non-linear functions, except that the activation here is stochastic and following some random distribution. Part I of the work focuses on the exact MMSE estimator, whose implementation is long known infeasible. For this exact estimator, an asymptotic analysis on the performance is carried out using a new replica method that is refined from certain aspects. A decoupling principle is then established, suggesting that, in terms of joint input-and-estimate distribution, the original estimation problem of multiple-input multiple-output is indeed identical to a simple single-input single-output one subjected to additive white Gaussian noise (AWGN) only. The variance of the AWGN is further shown to be determined by some coupled equations, whose dependency on the weighting and activation is given explicitly and analytically. Comparing to existing results, this paper is the first to offer a decoupling principle for the ML-GLM estimation problem. To further address the implementation issue of an exact solution, Part II proposes an approximate estimator, ML-GAMP, whose per-iteration complexity is as low as GAMP, while its asymptotic MSE (if converged) is as optimal as the exact MMSE estimator.
قيم البحث
اقرأ أيضاً
This is Part II of a two-part work on the estimation for a multi-layer generalized linear model (ML-GLM) in large system limits. In Part I, we had analyzed the asymptotic performance of an exact MMSE estimator, and obtained a set of coupled equations
that could characterize its MSE performance. To work around the implementation difficulty of the exact estimator, this paper continues to propose an approximate solution, ML-GAMP, which could be derived by blending a moment-matching projection into the Gaussian approximated loopy belief propagation. The ML-GAMP estimator is then shown to enjoy a great simplicity in its implementation, where its per-iteration complexity is as low as GAMP. Further analysis on its asymptotic performance also reveals that, in large system limits, its dynamical MSE behavior is fully characterized by a set of simple one-dimensional iterating equations, termed state evolution (SE). Interestingly, this SE of ML-GAMP share exactly the same fixed points with an exact MMSE estimator whose fixed points were obtained in Part I via a replica analysis. Given the Bayes-optimality of the exact implementation, this proposed estimator (if converged) is optimal in the MSE sense.
This paper extends the single crossing point property of the scalar MMSE function, derived by Guo, Shamai and Verdu (first presented in ISIT 2008), to the parallel degraded MIMO scenario. It is shown that the matrix Q(t), which is the difference betw
een the MMSE assuming a Gaussian input and the MMSE assuming an arbitrary input, has, at most, a single crossing point for each of its eigenvalues. Together with the I-MMSE relationship, a fundamental connection between Information Theory and Estimation Theory, this new property is employed to derive results in Information Theory. As a simple application of this property we provide an alternative converse proof for the broadcast channel (BC) capacity region under covariance constraint in this specific setting.
The scalar additive Gaussian noise channel has the single crossing point property between the minimum-mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE assuming an
arbitrary input. This paper extends the result to the parallel MIMO additive Gaussian channel in three phases: i) The channel matrix is the identity matrix, and we limit the Gaussian input to a vector of Gaussian i.i.d. elements. The single crossing point property is with respect to the snr (as in the scalar case). ii) The channel matrix is arbitrary, the Gaussian input is limited to an independent Gaussian input. A single crossing point property is derived for each diagonal element of the MMSE matrix. iii) The Gaussian input is allowed to be an arbitrary Gaussian random vector. A single crossing point property is derived for each eigenvalue of the MMSE matrix. These three extensions are then translated to new information theoretic properties on the mutual information, using the fundamental relationship between estimation theory and information theory. The results of the last phase are also translated to a new property of Fishers information. Finally, the applicability of all three extensions on information theoretic problems is demonstrated through: a proof of a special case of Shannons vector EPI, a converse proof of the capacity region of the parallel degraded MIMO broadcast channel (BC) under per-antenna power constrains and under covariance constraints, and a converse proof of the capacity region of the compound parallel degraded MIMO BC under covariance constraint.
We study a two-user state-dependent generalized multiple-access channel (GMAC) with correlated states. It is assumed that each encoder has emph{noncausal} access to channel state information (CSI). We develop an achievable rate region by employing ra
te-splitting, block Markov encoding, Gelfand--Pinsker multicoding, superposition coding and joint typicality decoding. In the proposed scheme, the encoders use a partial decoding strategy to collaborate in the next block, and the receiver uses a backward decoding strategy with joint unique decoding at each stage. Our achievable rate region includes several previously known regions proposed in the literature for different scenarios of multiple-access and relay channels. Then, we consider two Gaussian GMACs with additive interference. In the first model, we assume that the interference is known noncausally at both of the encoders and construct a multi-layer Costa precoding scheme that removes emph{completely} the effect of the interference. In the second model, we consider a doubly dirty Gaussian GMAC in which each of interferences is known noncausally only at one encoder. We derive an inner bound and analyze the achievable rate region for the latter model and interestingly prove that if one of the encoders knows the full CSI, there exists an achievable rate region which is emph{independent} of the power of interference.
In this paper, we extend the bilinear generalized approximate message passing (BiG-AMP) approach, originally proposed for high-dimensional generalized bilinear regression, to the multi-layer case for the handling of cascaded problem such as matrix-fa
ctorization problem arising in relay communication among others. Assuming statistically independent matrix entries with known priors, the new algorithm called ML-BiGAMP could approximate the general sum-product loopy belief propagation (LBP) in the high-dimensional limit enjoying a substantial reduction in computational complexity. We demonstrate that, in large system limit, the asymptotic MSE performance of ML-BiGAMP could be fully characterized via a set of simple one-dimensional equations termed state evolution (SE). We establish that the asymptotic MSE predicted by ML-BiGAMP SE matches perfectly the exact MMSE predicted by the replica method, which is well known to be Bayes-optimal but infeasible in practice. This consistency indicates that the ML-BiGAMP may still retain the same Bayes-optimal performance as the MMSE estimator in high-dimensional applications, although ML-BiGAMPs computational burden is far lower. As an illustrative example of the general ML-BiGAMP, we provide a detector design that could estimate the channel fading and the data symbols jointly with high precision for the two-hop amplify-and-forward relay communication systems.
سجل دخول لتتمكن من نشر تعليقات
التعليقات
جاري جلب التعليقات
سجل دخول لتتمكن من متابعة معايير البحث التي قمت باختيارها
