Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userLearning Cross-lingual Representations for Event Coreference Resolution with Multi-view Alignment and Optimal Transport
تعلم تمثيلات عبر اللغات لتحليل Aquerence الحدث مع محاذاة متعددة الرؤية والنقل الأمثل
445
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
ندرس مشكلة جديدة في التعلم عبر التحويلات المتبادلة لحدث القرار (ECR) حيث يتم تكييف النماذج المدربة على البيانات من لغة مصدر للتقييمات باللغات المستهدفة المختلفة. نقدم النموذج الأساسي الأول لهذه المهمة بناء على نموذج لغة XLM-Roberta، وهو نموذج لغوي متعدد اللغات مسبقا. نحن نستكشف أيضا الشبكات العصبية اللغوية اللغوية (LANN) التي تتولى التمييز بين النصوص من المصدر واللغات المستهدفة لتحسين تعميم اللغة ل ECR. بالإضافة إلى ذلك، نقدم آليتين رواية لتعزيز التعلم التمثيلي العام ل LANN، والتي تتميز بما يلي: (1) محاذاة متعددة الرؤية لمعاقبة محاذاة التسمية العاصمة من Aquerence من الأمثلة في المصدر واللغات المستهدفة، و (2) النقل الأمثل إلى حدد أمثلة وثيقة في المصدر واللغات المستهدفة لتوفير إشارات تدريبية أفضل لتمييز اللغة. أخيرا، نقوم بإجراء تجارب مكثفة ل ECR عبر اللغات من الإنجليزية إلى الإسبانية والصينية لإظهار فعالية الأساليب المقترحة.
We study a new problem of cross-lingual transfer learning for event coreference resolution (ECR) where models trained on data from a source language are adapted for evaluations in different target languages. We introduce the first baseline model for this task based on XLM-RoBERTa, a state-of-the-art multilingual pre-trained language model. We also explore language adversarial neural networks (LANN) that present language discriminators to distinguish texts from the source and target languages to improve the language generalization for ECR. In addition, we introduce two novel mechanisms to further enhance the general representation learning of LANN, featuring: (i) multi-view alignment to penalize cross coreference-label alignment of examples in the source and target languages, and (ii) optimal transport to select close examples in the source and target languages to provide better training signals for the language discriminators. Finally, we perform extensive experiments for cross-lingual ECR from English to Spanish and Chinese to demonstrate the effectiveness of the proposed methods.
References used
https://aclanthology.org/
rate research
Read More
In this paper, we propose to align sentence representations from different languages into a unified embedding space, where semantic similarities (both cross-lingual and monolingual) can be computed with a simple dot product. Pre-trained language mode
ls are fine-tuned with the translation ranking task. Existing work (Feng et al., 2020) uses sentences within the same batch as negatives, which can suffer from the issue of easy negatives. We adapt MoCo (He et al., 2020) to further improve the quality of alignment. As the experimental results show, the sentence representations produced by our model achieve the new state-of-the-art on several tasks, including Tatoeba en-zh similarity search (Artetxe andSchwenk, 2019b), BUCC en-zh bitext mining, and semantic textual similarity on 7 datasets.
Recent studies have proposed different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. For contextualized embeddings, alignment becomes more c
omplex as we additionally take context into consideration. In this work, we propose using Optimal Transport (OT) as an alignment objective during fine-tuning to further improve multilingual contextualized representations for downstream cross-lingual transfer. This approach does not require word-alignment pairs prior to fine-tuning that may lead to sub-optimal matching and instead learns the word alignments within context in an unsupervised manner. It also allows different types of mappings due to soft matching between source and target sentences. We benchmark our proposed method on two tasks (XNLI and XQuAD) and achieve improvements over baselines as well as competitive results compared to similar recent works.
Pretrained multilingual language models have become a common tool in transferring NLP capabilities to low-resource languages, often with adaptations. In this work, we study the performance, extensibility, and interaction of two such adaptations: voca
bulary augmentation and script transliteration. Our evaluations on part-of-speech tagging, universal dependency parsing, and named entity recognition in nine diverse low-resource languages uphold the viability of these approaches while raising new questions around how to optimally adapt multilingual models to low-resource settings.
We consider the problem of learning to repair erroneous C programs by learning optimal alignments with correct programs. Since the previous approaches fix a single error in a line, it is inevitable to iterate the fixing process until no errors remain
. In this work, we propose a novel sequence-to-sequence learning framework for fixing multiple program errors at a time. We introduce the edit-distance-based data labeling approach for program error correction. Instead of labeling a program repair example by pairing an erroneous program with a line fix, we label the example by paring an erroneous program with an optimal alignment to the corresponding correct program produced by the edit-distance computation. We evaluate our proposed approach on a publicly available dataset (DeepFix dataset) that consists of erroneous C programs submitted by novice programming students. On a set of 6,975 erroneous C programs from the DeepFix dataset, our approach achieves the state-of-the-art result in terms of full repair rate on the DeepFix dataset (without extra data such as compiler error message or additional source codes for pre-training).
Recent literatures have shown that knowledge graph (KG) learning models are highly vulnerable to adversarial attacks. However, there is still a paucity of vulnerability analyses of cross-lingual entity alignment under adversarial attacks. This paper
proposes an adversarial attack model with two novel attack techniques to perturb the KG structure and degrade the quality of deep cross-lingual entity alignment. First, an entity density maximization method is employed to hide the attacked entities in dense regions in two KGs, such that the derived perturbations are unnoticeable. Second, an attack signal amplification method is developed to reduce the gradient vanishing issues in the process of adversarial attacks for further improving the attack effectiveness.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
