Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAdversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
79
0
0.0
(
0
)
Added by
Simiao Zuo
Publication date
2021
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks. Existing works usually formulate the method as a zero-sum game, which is solved by alternating gradient descent/ascent algorithms. Such a formulation treats the adversarial and the defending players equally, which is undesirable because only the defending player contributes to the generalization performance. To address this issue, we propose Stackelberg Adversarial Regularization (SALT), which formulates adversarial regularization as a Stackelberg game. This formulation induces a competition between a leader and a follower, where the follower generates perturbations, and the leader trains the model subject to the perturbations. Different from conventional approaches, in SALT, the leader is in an advantageous position. When the leader moves, it recognizes the strategy of the follower and takes the anticipated followers outcomes into consideration. Such a leaders advantage enables us to improve the model fitting to the unperturbed data. The leaders strategic information is captured by the Stackelberg gradient, which is obtained using an unrolling algorithm. Our experimental results on a set of machine translation and natural language understanding tasks show that SALT outperforms existing adversarial regularization baselines across all tasks. Our code is publicly available.
rate research
Read More
Understanding procedural text requires tracking entities, actions and effects as the narrative unfolds. We focus on the challenging real-world problem of action-graph extraction from material science papers, where language is highly specialized and data annotation is expensive and scarce. We propose a novel approach, Text2Quest, where procedural text is interpreted as instructions for an interactive game. A learning agent completes the game by executing the procedure correctly in a text-based simulated lab environment. The framework can complement existing approaches and enables richer forms of learning compared to static texts. We discuss potential limitations and advantages of the approach, and release a prototype proof-of-concept, hoping to encourage research in this direction.
It is known that the capacity of the intelligent reflecting surface (IRS) aided cellular network can be effectively improved by reflecting the incident signals from the transmitter in a low-cost passive reflecting way. Nevertheless, in the actual network operation, the base station (BS) and IRS may belong to different operators, consequently, the IRS is reluctant to help the BS without any payment. Therefore, this paper investigates price-based reflection resource (elements) allocation strategies for an IRS-aided multiuser multiple-input and single-output (MISO) downlink communication systems, in which all transmissions over the same frequency band. Assuming that the IRS is composed with multiple modules, each of which is attached with a smart controller, thus, the states (active/idle) of module can be operated by its controller, and all controllers can be communicated with each other via fiber links. A Stackelberg game-based alternating direction method of multipliers (ADMM) is proposed to jointly optimize the transmit beamforming at the BS and the passive beamforming of the active modules. Numerical examples are presented to verify the proposed algorithm. It is shown that the proposed scheme is effective in the utilities of both the BS and IRS.
Fog computing is a promising architecture to provide economic and low latency data services for future Internet of things (IoT)-based network systems. It relies on a set of low-power fog nodes that are close to the end users to offload the services originally targeting at cloud data centers. In this paper, we consider a specific fog computing network consisting of a set of data service operators (DSOs) each of which controls a set of fog nodes to provide the required data service to a set of data service subscribers (DSSs). How to allocate the limited computing resources of fog nodes (FNs) to all the DSSs to achieve an optimal and stable performance is an important problem. In this paper, we propose a joint optimization framework for all FNs, DSOs and DSSs to achieve the optimal resource allocation schemes in a distributed fashion. In the framework, we first formulate a Stackelberg game to analyze the pricing problem for the DSOs as well as the resource allocation problem for the DSSs. Under the scenarios that the DSOs can know the expected amount of resource purchased by the DSSs, a many-to-many matching game is applied to investigate the pairing problem between DSOs and FNs. Finally, within the same DSO, we apply another layer of many-to-many matching between each of the paired FNs and serving DSSs to solve the FN-DSS pairing problem. Simulation results show that our proposed framework can significantly improve the performance of the IoT-based network systems.
A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model. Traditionally, hand-crafted priors along with iterative optimization methods have been used to solve such problems. In this paper we present unrolled optimization with deep priors, a principled framework for infusing knowledge of the image formation into deep networks that solve inverse problems in imaging, inspired by classical iterative methods. We show that instances of the framework outperform the state-of-the-art by a substantial margin for a wide variety of imaging problems, such as denoising, deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover, we conduct experiments that explain how the framework is best used and why it outperforms previous methods.
State-of-the-art NLP models can often be fooled by human-unaware transformations such as synonymous word substitution. For security reasons, it is of critical importance to develop models with certified robustness that can provably guarantee that the prediction is can not be altered by any possible synonymous word substitution. In this work, we propose a certified robust method based on a new randomized smoothing technique, which constructs a stochastic ensemble by applying random word substitutions on the input sentences, and leverage the statistical properties of the ensemble to provably certify the robustness. Our method is simple and structure-free in that it only requires the black-box queries of the model outputs, and hence can be applied to any pre-trained models (such as BERT) and any types of models (world-level or subword-level). Our method significantly outperforms recent state-of-the-art methods for certified robustness on both IMDB and Amazon text classification tasks. To the best of our knowledge, we are the first work to achieve certified robustness on large systems such as BERT with practically meaningful certified accuracy.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
