Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userImproving Stance Detection with Multi-Dataset Learning and Knowledge Distillation
تحسين اكتشاف الموقف مع التعلم متعدد البيانات والتقطير المعرفي
290
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يحدد اكتشاف الموقف ما إذا كان مؤلف النص مؤهلا لصالح أو محايد هدف معين ويوفر رؤى قيمة في أحداث مهمة مثل تقنين الإجهاض. على الرغم من التقدم الكبير في هذه المهمة، فإن أحد التحديات المتبقية هو ندرة التعليقات التوضيحية. علاوة على ذلك، ركزت معظم الأعمال السابقة على تدريبا ثابتا على التسمية التي يتم فيها التخلص منها تشابه ذات معنى بين الفئات أثناء التدريب. لمعالجة هذه التحديات أولا، نقيم هدف متعدد المستهدف وإعدادات تدريب متعددة البيانات من خلال تدريب نموذج واحد على كل مجموعة بيانات ومجموعات من المجالات المختلفة، على التوالي. نظهر أن النماذج يمكن أن تتعلم المزيد من التمثيلات العالمية فيما يتعلق بالأهداف في هذه الإعدادات. ثانيا، يمكننا التحقيق في تقطير المعرفة في اكتشاف الموقف ومراقبة أن نقل المعرفة من نموذج المعلم إلى نموذج الطالب يمكن أن يكون مفيدا في إعدادات التدريب المقترحة. علاوة على ذلك، نقترح طريقة تقطير المعرفة التكيفية (AKD) تطبق تحجيم درجة الحرارة الخاصة بالمثيلات إلى المعلم والتنبؤات الطلابية. تشير النتائج إلى أن نموذج متعدد البيانات يعمل بشكل أفضل على جميع مجموعات البيانات ويمكن تحسينه من قبل AKD المقترح، مما يتفوق على أحدث حالة من الهامش الكبير. نحن نطلق علنا كودنا.
Stance detection determines whether the author of a text is in favor of, against or neutral to a specific target and provides valuable insights into important events such as legalization of abortion. Despite significant progress on this task, one of the remaining challenges is the scarcity of annotations. Besides, most previous works focused on a hard-label training in which meaningful similarities among categories are discarded during training. To address these challenges, first, we evaluate a multi-target and a multi-dataset training settings by training one model on each dataset and datasets of different domains, respectively. We show that models can learn more universal representations with respect to targets in these settings. Second, we investigate the knowledge distillation in stance detection and observe that transferring knowledge from a teacher model to a student model can be beneficial in our proposed training settings. Moreover, we propose an Adaptive Knowledge Distillation (AKD) method that applies instance-specific temperature scaling to the teacher and student predictions. Results show that the multi-dataset model performs best on all datasets and it can be further improved by the proposed AKD, outperforming the state-of-the-art by a large margin. We publicly release our code.
References used
https://aclanthology.org/
rate research
Read More
Curriculum learning, a machine training strategy that feeds training instances to the model from easy to hard, has been proven to facilitate the dialogue generation task. Meanwhile, knowledge distillation, a knowledge transformation methodology among
teachers and students networks can yield significant performance boost for student models. Hence, in this paper, we introduce a combination of curriculum learning and knowledge distillation for efficient dialogue generation models, where curriculum learning can help knowledge distillation from data and model aspects. To start with, from the data aspect, we cluster the training cases according to their complexity, which is calculated by various types of features such as sentence length and coherence between dialog pairs. Furthermore, we employ an adversarial training strategy to identify the complexity of cases from model level. The intuition is that, if a discriminator can tell the generated response is from the teacher or the student, then the case is difficult that the student model has not adapted to yet. Finally, we use self-paced learning, which is an extension to curriculum learning to assign weights for distillation. In conclusion, we arrange a hierarchical curriculum based on the above two aspects for the student model under the guidance from the teacher model. Experimental results demonstrate that our methods achieve improvements compared with competitive baselines.
Modelling and understanding dialogues in a conversation depends on identifying the user intent from the given text. Unknown or new intent detection is a critical task, as in a realistic scenario a user intent may frequently change over time and diver
t even to an intent previously not encountered. This task of separating the unknown intent samples from known intents one is challenging as the unknown user intent can range from intents similar to the predefined intents to something completely different. Prior research on intent discovery often consider it as a classification task where an unknown intent can belong to a predefined set of known intent classes. In this paper we tackle the problem of detecting a completely unknown intent without any prior hints about the kind of classes belonging to unknown intents. We propose an effective post-processing method using multi-objective optimization to tune an existing neural network based intent classifier and make it capable of detecting unknown intents. We perform experiments using existing state-of-the-art intent classifiers and use our method on top of them for unknown intent detection. Our experiments across different domains and real-world datasets show that our method yields significant improvements compared with the state-of-the-art methods for unknown intent detection.
We study the problem of performing automatic stance classification on social media with neural architectures such as BERT. Although these architectures deliver impressive results, their level is not yet comparable to the one of humans and they might
produce errors that have a significant impact on the downstream task (e.g., fact-checking). To improve the performance, we present a new neural architecture where the input also includes automatically generated negated perspectives over a given claim. The model is jointly learned to make simultaneously multiple predictions, which can be used either to improve the classification of the original perspective or to filter out doubtful predictions. In the first case, we propose a weakly supervised method for combining the predictions into a final one. In the second case, we show that using the confidence scores to remove doubtful predictions allows our method to achieve human-like performance over the retained information, which is still a sizable part of the original input.
The goal of stance detection is to identify whether the author of a text is in favor of, neutral or against a specific target. Despite substantial progress on this task, one of the remaining challenges is the scarcity of annotations. Data augmentatio
n is commonly used to address annotation scarcity by generating more training samples. However, the augmented sentences that are generated by existing methods are either less diversified or inconsistent with the given target and stance label. In this paper, we formulate the data augmentation of stance detection as a conditional masked language modeling task and augment the dataset by predicting the masked word conditioned on both its context and the auxiliary sentence that contains target and label information. Moreover, we propose another simple yet effective method that generates target-aware sentence by replacing a target mention with the other. Experimental results show that our proposed methods significantly outperforms previous augmentation methods on 11 targets.
Detecting stance on Twitter is especially challenging because of the short length of each tweet, the continuous coinage of new terminology and hashtags, and the deviation of sentence structure from standard prose. Fine-tuned language models using lar
ge-scale in-domain data have been shown to be the new state-of-the-art for many NLP tasks, including stance detection. In this paper, we propose a novel BERT-based fine-tuning method that enhances the masked language model for stance detection. Instead of random token masking, we propose using a weighted log-odds-ratio to identify words with high stance distinguishability and then model an attention mechanism that focuses on these words. We show that our proposed approach outperforms the state of the art for stance detection on Twitter data about the 2020 US Presidential election.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
