Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUnsupervised Domain Adaptation Method with Semantic-Structural Alignment for Dependency Parsing
طريقة تكيف المجال غير المزعجة مع محاذاة الهيكلية الدلالية لتحليل التبعية
490
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تحليل التبعية عبر المجال غير الخاضع للإكمال هو إنجاز تكيف مجال تحليل التبعية دون استخدام البيانات المسمى في المجال المستهدف. غالبا ما تكون الأساليب الحالية من نوع التوضيح الزائفة، والتي تنشئ البيانات من خلال التوضيح الذاتي للنموذج الأساسي وأداء التدريب التكراري. ومع ذلك، فشلت هذه الطرق في النظر في تغيير هيكل النموذج لتكييف المجال. بالإضافة إلى ذلك، لا يمكن استغلال المعلومات الهيكلية الواردة في النص بالكامل. لعلاج هذه العيوب، نقترح محلل التبعية التبعية للتكيف مع بنية دلالات (SSADP)، التي تنجز تحليلات التبعية عبر المجال غير الخاضعة للكشف دون الاعتماد على التوضيح الزائفة أو اختيار البيانات. على وجه الخصوص، نقوم بتصميم اثنين من النازعين ميزة لاستخراج الميزات الدلالية والهيكلية على التوالي. لكل نوع من الميزات، يتم استخدام طريقة تكيف الميزة المقابلة لتحقيق تكيف المجال لمواءمة توزيع المجال، والتي تعزز بشكل فعال إمكانية نقل المجال المتقاطع بشكل فعال للنموذج. نحن نقوم بالتحقق من فعالية طرازنا عن طريق إجراء تجارب على Codt1 و CTB9 على التوالي، وتظهر النتائج أن نموذجنا يمكن أن يحقق تحسين أداء ثابتا. علاوة على ذلك، نتحقق من قدرة نقل الهيكل النموذج المقترح عن طريق إدخال اختبار Weisfeiler-Lehman.
Unsupervised cross-domain dependency parsing is to accomplish domain adaptation for dependency parsing without using labeled data in target domain. Existing methods are often of the pseudo-annotation type, which generates data through self-annotation of the base model and performing iterative training. However, these methods fail to consider the change of model structure for domain adaptation. In addition, the structural information contained in the text cannot be fully exploited. To remedy these drawbacks, we propose a Semantics-Structure Adaptative Dependency Parser (SSADP), which accomplishes unsupervised cross-domain dependency parsing without relying on pseudo-annotation or data selection. In particular, we design two feature extractors to extract semantic and structural features respectively. For each type of features, a corresponding feature adaptation method is utilized to achieve domain adaptation to align the domain distribution, which effectively enhances the unsupervised cross-domain transfer capability of the model. We validate the effectiveness of our model by conducting experiments on the CODT1 and CTB9 respectively, and the results demonstrate that our model can achieve consistent performance improvement. Besides, we verify the structure transfer ability of the proposed model by introducing Weisfeiler-Lehman Test.
References used
https://aclanthology.org/
rate research
Read More
This paper presents the Source-Free Domain Adaptation shared task held within SemEval-2021. The aim of the task was to explore adaptation of machine-learning models in the face of data sharing constraints. Specifically, we consider the scenario where
annotations exist for a domain but cannot be shared. Instead, participants are provided with models trained on that (source) data. Participants also receive some labeled data from a new (development) domain on which to explore domain adaptation algorithms. Participants are then tested on data representing a new (target) domain. We explored this scenario with two different semantic tasks: negation detection (a text classification task) and time expression recognition (a sequence tagging task).
This paper considers the unsupervised domain adaptation problem for neural machine translation (NMT), where we assume the access to only monolingual text in either the source or target language in the new domain. We propose a cross-lingual data selec
tion method to extract in-domain sentences in the missing language side from a large generic monolingual corpus. Our proposed method trains an adaptive layer on top of multilingual BERT by contrastive learning to align the representation between the source and target language. This then enables the transferability of the domain classifier between the languages in a zero-shot manner. Once the in-domain data is detected by the classifier, the NMT model is then adapted to the new domain by jointly learning translation and domain discrimination tasks. We evaluate our cross-lingual data selection method on NMT across five diverse domains in three language pairs, as well as a real-world scenario of translation for COVID-19. The results show that our proposed method outperforms other selection baselines up to +1.5 BLEU score.
The state-of-the-art abusive language detection models report great in-corpus performance, but underperform when evaluated on abusive comments that differ from the training scenario. As human annotation involves substantial time and effort, models th
at can adapt to newly collected comments can prove to be useful. In this paper, we investigate the effectiveness of several Unsupervised Domain Adaptation (UDA) approaches for the task of cross-corpora abusive language detection. In comparison, we adapt a variant of the BERT model, trained on large-scale abusive comments, using Masked Language Model (MLM) fine-tuning. Our evaluation shows that the UDA approaches result in sub-optimal performance, while the MLM fine-tuning does better in the cross-corpora setting. Detailed analysis reveals the limitations of the UDA approaches and emphasizes the need to build efficient adaptation methods for this task.
In this paper, we propose a simple few-shot domain adaptation paradigm for reading comprehension. We first identify the lottery subnetwork structure within the Transformer-based source domain model via gradual magnitude pruning. Then, we only fine-tu
ne the lottery subnetwork, a small fraction of the whole parameters, on the annotated target domain data for adaptation. To obtain more adaptable subnetworks, we introduce self-attention attribution to weigh parameters, beyond simply pruning the smallest magnitude parameters, which can be seen as combining structured pruning and unstructured magnitude pruning softly. Experimental results show that our method outperforms the full model fine-tuning adaptation on four out of five domains when only a small amount of annotated data available for adaptation. Moreover, introducing self-attention attribution reserves more parameters for important attention heads in the lottery subnetwork and improves the target domain model performance. Our further analyses reveal that, besides exploiting fewer parameters, the choice of subnetworks is critical to the effectiveness.
Recent work has demonstrated that pre-training in-domain language models can boost performance when adapting to a new domain. However, the costs associated with pre-training raise an important question: given a fixed budget, what steps should an NLP
practitioner take to maximize performance? In this paper, we study domain adaptation under budget constraints, and approach it as a customer choice problem between data annotation and pre-training. Specifically, we measure the annotation cost of three procedural text datasets and the pre-training cost of three in-domain language models. Then we evaluate the utility of different combinations of pre-training and data annotation under varying budget constraints to assess which combination strategy works best. We find that, for small budgets, spending all funds on annotation leads to the best performance; once the budget becomes large enough, a combination of data annotation and in-domain pre-training works more optimally. We therefore suggest that task-specific data annotation should be part of an economical strategy when adapting an NLP model to a new domain.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
