Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userGramian-Based Adaptive Combination Policies for Diffusion Learning over Networks
58
0
0.0
(
0
)
Added by
Y. Efe Erginbas
Publication date
2020
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
This paper presents an adaptive combination strategy for distributed learning over diffusion networks. Since learning relies on the collaborative processing of the stochastic information at the dispersed agents, the overall performance can be improved by designing combination policies that adjust the weights according to the quality of the data. Such policies are important because they would add a new degree of freedom and endow multi-agent systems with the ability to control the flow of information over their edges for enhanced performance. Most adaptive and static policies available in the literature optimize certain performance metrics related to steady-state behavior, to the detriment of transient behavior. In contrast, we develop an adaptive combination rule that aims at optimizing the transient learning performance, while maintaining the enhanced steady-state performance obtained using policies previously developed in the literature.
rate research
Read More
Adaptive networks have the capability to pursue solutions of global stochastic optimization problems by relying only on local interactions within neighborhoods. The diffusion of information through repeated interactions allows for globally optimal behavior, without the need for central coordination. Most existing strategies are developed for cooperative learning settings, where the objective of the network is common to all agents. We consider in this work a team setting, where a subset of the agents form a team with a common goal while competing with the remainder of the network. We develop an algorithm for decentralized competition among teams of adaptive agents, analyze its dynamics and present an application in the decentralized training of generative adversarial neural networks.
The present work introduces the hybrid consensus alternating direction method of multipliers (H-CADMM), a novel framework for optimization over networks which unifies existing distributed optimization approaches, including the centralized and the decentralized consensus ADMM. H-CADMM provides a flexible tool that leverages the underlying graph topology in order to achieve a desirable sweet-spot between node-to-node communication overhead and rate of convergence -- thereby alleviating known limitations of both C-CADMM and D-CADMM. A rigorous analysis of the novel method establishes linear convergence rate, and also guides the choice of parameters to optimize this rate. The novel hybrid update rules of H-CADMM lend themselves to in-network acceleration that is shown to effect considerable -- and essentially free-of-charge -- performance boost over the fully decentralized ADMM. Comprehensive numerical tests validate the analysis and showcase the potential of the method in tackling efficiently, widely useful learning tasks.
In this work we analyze the mean-square performance of different strategies for distributed estimation over least-mean-squares (LMS) adaptive networks. The results highlight some useful properties for distributed adaptation in comparison to fusion-based centralized solutions. The analysis establishes that, by optimizing over the combination weights, diffusion strategies can deliver lower excess-mean-square-error than centralized solutions employing traditional block or incremental LMS strategies. We first study in some detail the situation involving combinations of two adaptive agents and then extend the results to generic N-node ad-hoc networks. In the later case, we establish that, for sufficiently small step-sizes, diffusion strategies can outperform centralized block or incremental LMS strategies by optimizing over left-stochastic combination weighting matrices. The results suggest more efficient ways for organizing and processing data at fusion centers, and present useful adaptive strategies that are able to enhance performance when implemented in a distributed manner.
This paper investigates a model reduction problem for linear directed network systems, in which the interconnections among the vertices are described by general weakly connected digraphs. First, the definitions of pseudo controllability and observability Gramians are proposed for semistable systems, and their solutions are characterized by Lyapunov-like equations. Then, we introduce a concept of vertex clusterability to guarantee the boundedness of the approximation error and use the newly proposed Gramians to facilitate the evaluation of the dissimilarity of each pair of vertices. An clustering algorithm is thereto provided to generate an appropriate graph clustering, whose characteristic matrix is employed as the projections in the Petrov-Galerkin reduction framework. The obtained reduced-order system preserves the weakly connected directed network structure, and the approximation error is computed by the pseudo Gramians. Finally, the efficiency of the proposed approach is illustrated by numerical examples.
In this paper, we compare four measures of the empirical observability gramian, including the determinant, the trace, the minimum eigenvalue, and the condition number, which can be used to quantify the observability of system states and to obtain the optimal PMU placement for power system dynamic state estimation. An adaptive optimal PMU placement method is proposed by automatically choosing proper measures as the objective function. It is shown that when the number of PMUs is small and thus the observability is very weak, the minimum eigenvalue and the condition number are better measures of the observability and are preferred to be chosen as the objective function. The effectiveness of the proposed method is validated by performing dynamic state estimation on an Northeast Power Coordinating Council (NPCC) 48-machine 140-bus system with the square-root unscented Kalman filter.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
