Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAdversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
التنظيم الخصم كما لعبة Stackelberg: نهج التحسين غير المنصوص عليه
90
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
وقد تبين أن التنظيم العديزي لتحسين أداء تعميم نماذج التعلم العميق في مهام معالجة اللغة الطبيعية المختلفة. تعمل الأعمال الموجودة عادة الطريقة كأفضل لعبة مبلغ صفر، والتي تم حلها من خلال خوارزميات نزول / صعود التدرج المتناوب. مثل هذه الصياغة يعامل اللاعبين والدفاع عن اللاعبين على قدم المساواة، وهو أمر غير مرغوب فيه لأن اللاعب المدافع فقط يساهم في أداء التعميم. لمعالجة هذه المسألة، نقترح بنظام Stackelberg الخصم (الملح)، الذي يصوغ التنظيم العديزي كأرعاب Stackelberg. يستحث هذا الصيغة منافسة بين قائد ومتابعته، حيث يولد التابع الاضطرابات، والقائد يدرب النموذج المعني بالاضطرابات. تختلف عن الأساليب التقليدية، في السلط، الزعيم في وضع مفيد. عندما يتحرك القائد، فإنه يتعرف على استراتيجية التابع ويأخذ نتائج التابع المتوقعة في الاعتبار. تمكننا ميزة الزعيم هذه من تحسين النموذج المناسب للبيانات غير المضطربة. يتم التقاط المعلومات الاستراتيجية للزعيم من قبل التدرج من Stackelberg، والتي يتم الحصول عليها باستخدام خوارزمية غير مثيرة. تظهر نتائجنا التجريبية على مجموعة من الترجمة الآلية ومهام فهم اللغة الطبيعية أن الملح يتفوق على خطوط خطوط الأساس بين المخدرات الموجودة في جميع المهام. رمز لدينا هو متاح علنا.
Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks. Existing works usually formulate the method as a zero-sum game, which is solved by alternating gradient descent/ascent algorithms. Such a formulation treats the adversarial and the defending players equally, which is undesirable because only the defending player contributes to the generalization performance. To address this issue, we propose Stackelberg Adversarial Regularization (SALT), which formulates adversarial regularization as a Stackelberg game. This formulation induces a competition between a leader and a follower, where the follower generates perturbations, and the leader trains the model subject to the perturbations. Different from conventional approaches, in SALT, the leader is in an advantageous position. When the leader moves, it recognizes the strategy of the follower and takes the anticipated follower's outcomes into consideration. Such a leader's advantage enables us to improve the model fitting to the unperturbed data. The leader's strategic information is captured by the Stackelberg gradient, which is obtained using an unrolling algorithm. Our experimental results on a set of machine translation and natural language understanding tasks show that SALT outperforms existing adversarial regularization baselines across all tasks. Our code is publicly available.
References used
https://aclanthology.org/
rate research
Read More
We investigate if a model can learn natural language with minimal linguistic input through interaction. Addressing this question, we design and implement an interactive language learning game that learns logical semantic representations compositional
ly. Our game allows us to explore the benefits of logical inference for natural language learning. Evaluation shows that the model can accurately narrow down potential logical representations for words over the course of the game, suggesting that our model is able to learn lexical mappings from scratch successfully.
In this paper, we address unsupervised chunking as a new task of syntactic structure induction, which is helpful for understanding the linguistic structures of human languages as well as processing low-resource languages. We propose a knowledge-trans
fer approach that heuristically induces chunk labels from state-of-the-art unsupervised parsing models; a hierarchical recurrent neural network (HRNN) learns from such induced chunk labels to smooth out the noise of the heuristics. Experiments show that our approach largely bridges the gap between supervised and unsupervised chunking.
Online reviews are an essential aspect of online shopping for both customers and retailers. However, many reviews found on the Internet lack in quality, informativeness or helpfulness. In many cases, they lead the customers towards positive or negati
ve opinions without providing any concrete details (e.g., very poor product, I would not recommend it). In this work, we propose a novel unsupervised method for quantifying helpfulness leveraging the availability of a corpus of reviews. In particular, our method exploits three characteristics of the reviews, viz., relevance, emotional intensity and specificity, towards quantifying helpfulness. We perform three rankings (one for each feature above), which are then combined to obtain a final helpfulness ranking. For the purpose of empirically evaluating our method, we use review of four product categories from Amazon review. The experimental evaluation demonstrates the effectiveness of our method in comparison to a recent and state-of-the-art baseline.
Abstract Text classification is a widely studied problem and has broad applications. In many real-world problems, the number of texts for training classification models is limited, which renders these models prone to overfitting. To address this prob
lem, we propose SSL-Reg, a data-dependent regularization approach based on self-supervised learning (SSL). SSL (Devlin et al., 2019a) is an unsupervised learning approach that defines auxiliary tasks on input data without using any human-provided labels and learns data representations by solving these auxiliary tasks. In SSL-Reg, a supervised classification task and an unsupervised SSL task are performed simultaneously. The SSL task is unsupervised, which is defined purely on input texts without using any human- provided labels. Training a model using an SSL task can prevent the model from being overfitted to a limited number of class labels in the classification task. Experiments on 17 text classification datasets demonstrate the effectiveness of our proposed method. Code is available at https://github.com/UCSD-AI4H/SSReg.
Unsupervised relation extraction works by clustering entity pairs that have the same relations in the text. Some existing variational autoencoder (VAE)-based approaches train the relation extraction model as an encoder that generates relation classif
ications. A decoder is trained along with the encoder to reconstruct the encoder input based on the encoder-generated relation classifications. These classifications are a latent variable so they are required to follow a pre-defined prior distribution which results in unstable training. We propose a VAE-based unsupervised relation extraction technique that overcomes this limitation by using the classifications as an intermediate variable instead of a latent variable. Specifically, classifications are conditioned on sentence input, while the latent variable is conditioned on both the classifications and the sentence input. This allows our model to connect the decoder with the encoder without putting restrictions on the classification distribution; which improves training stability. Our approach is evaluated on the NYT dataset and outperforms state-of-the-art methods.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
