Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userIncentive Mechanism Design for Distributed Coded Machine Learning
226
0
0.0
(
0
)
Added by
Ningning Ding
Publication date
2020
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
A distributed machine learning platform needs to recruit many heterogeneous worker nodes to finish computation simultaneously. As a result, the overall performance may be degraded due to straggling workers. By introducing redundancy into computation, coded machine learning can effectively improve the runtime performance by recovering the final computation result through the first $k$ (out of the total $n$) workers who finish computation. While existing studies focus on designing efficient coding schemes, the issue of designing proper incentives to encourage worker participation is still under-explored. This paper studies the platforms optimal incentive mechanism for motivating proper workers participation in coded machine learning, despite the incomplete information about heterogeneous workers computation performances and costs. A key contribution of this work is to summarize workers multi-dimensional heterogeneity as a one-dimensional metric, which guides the platforms efficient selection of workers under incomplete information with a linear computation complexity. Moreover, we prove that the optimal recovery threshold $k$ is linearly proportional to the participator number $n$ if we use the widely adopted MDS (Maximum Distance Separable) codes for data encoding. We also show that the platforms increased cost due to incomplete information disappears when worker number is sufficiently large, but it does not monotonically decrease in worker number.
rate research
Read More
We describe a structured system for distributed mechanism design. It consists of a sequence of layers. The lower layers deal with the operations relevant for distributed computing only, while the upper layers are concerned only with communication among players, including broadcasting and multicasting, and distributed decision making. This yields a highly flexible distributed system whose specific applications are realized as instances of its top layer. This design supports fault-tolerance, prevents manipulations and makes it possible to implement distributed policing. The system is implemented in Java. We illustrate it by discussing a number of implemented examples.
In the standard Mechanism Design framework (Hurwicz-Reiter), there is a central authority that gathers agents messages and subsequently determines the allocation and tax for each agent. We consider a scenario where, due to communication overhead and other constraints, such broadcasting of messages to a central authority cannot take place. Instead, only local message exchange is allowed between agents. As a result, each agent should be able to determine her own allocation and tax based on the messages in the local neighborhood, as defined by a given message graph describing the communication constraints. This scenario gives rise to a novel research direction that we call Distributed Mechanism Design. In this paper, we propose such a distributed mechanism for the problem of rate allocation in a multicast transmission network. The proposed mechanism fully implements the optimal allocation in Nash equilibria and its message space dimension is linear with respect to the number of agents in the network.
Federated learning (FL) has shown great potential for addressing the challenge of isolated data islands while preserving data privacy. It allows artificial intelligence (AI) models to be trained on locally stored data in a distributed manner. In order to build an ecosystem for FL to operate in a sustainable manner, it has to be economically attractive to data owners. This gives rise to the problem of FL incentive mechanism design, which aims to find the optimal organizational and payment structure for the federation in order to achieve a series of economic objectives. In this paper, we present a VCG-based FL incentive mechanism, named FVCG, specifically designed for incentivizing data owners to contribute all their data and truthfully report their costs in FL settings. It maximizes the social surplus and minimizes unfairness of the federation. We provide an implementation of FVCG with neural networks and theoretic proofs on its performance bounds. Extensive numerical experiment results demonstrated the effectiveness and economic reasonableness of FVCG.
Decision-making systems increasingly orchestrate our world: how to intervene on the algorithmic components to build fair and equitable systems is therefore a question of utmost importance; one that is substantially complicated by the context-dependent nature of fairness and discrimination. Modern decision-making systems that involve allocating resources or information to people (e.g., school choice, advertising) incorporate machine-learned predictions in their pipelines, raising concerns about potential strategic behavior or constrained allocation, concerns usually tackled in the context of mechanism design. Although both machine learning and mechanism design have developed frameworks for addressing issues of fairness and equity, in some complex decision-making systems, neither framework is individually sufficient. In this paper, we develop the position that building fair decision-making systems requires overcoming these limitations which, we argue, are inherent to each field. Our ultimate objective is to build an encompassing framework that cohesively bridges the individual frameworks of mechanism design and machine learning. We begin to lay the ground work towards this goal by comparing the perspective each discipline takes on fair decision-making, teasing out the lessons each field has taught and can teach the other, and highlighting application domains that require a strong collaboration between these disciplines.
In the standard Mechanism Design framework, agents messages are gathered at a central point and allocation/tax functions are calculated in a centralized manner, i.e., as functions of all network agents messages. This requirement may cause communication and computation overhead and necessitates the design of mechanisms that alleviate this bottleneck. We consider a scenario where message transmission can only be performed locally so that the mechanism allocation/tax functions can be calculated in a decentralized manner. Each agent transmits messages to her local neighborhood, as defined by a given message-exchange network, and her allocation/tax functions are only functions of the available neighborhood messages. This scenario gives rise to a novel research problem that we call Distributed Mechanism Design. In this paper, we propose two distributed mechanisms for network utility maximization problems that involve private and public goods with competition and cooperation between agents. As a concrete example, we use the problems of rate allocation in networks with either unicast or multirate multicast transmission protocols. The proposed mechanism for each of the protocols fully implements the optimal allocation in Nash equilibria and its message space dimensionality scales linearly with respect to the number of agents in the network.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
