Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCrosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments
التعلم تحويل التعلم عن العلاقة واستخراج الأحداث عبر فئة Word ومحاذاة الفصل
238
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يعاني العمل السابق على علاقة Crosslingual واستخراج الأحداث (REE) من قضية التحيز أحادي الأحادي بسبب تدريب النماذج على بيانات اللغة المصدر فقط. تتمثل نهج التغلب على هذه المسألة في استخدام البيانات غير المستهلكة في اللغة المستهدفة لمساعدة محاذاة تمثيلات crosslingual، أي عن طريق خداع تمييز لغة. ومع ذلك، نظرا لأن هذا النهج غير بشرط على معلومات الفصل، فإن مثال لغوي مستهدف يمكن أن يتماشى بشكل غير صحيح إلى مثال لغة مصدر لفئة مختلفة. لمعالجة هذه المشكلة، نقترح طريقة محاذاة Crosslingual الجديدة التي ترفد معلومات الفئة من مهام REE لتعلم التمثيل. على وجه الخصوص، نقترح تعلم نسختين من ناقلات التمثيل لكل فصل في مهمة ري بناء على أمثلة اللغة أو الهدف المستهدف. سيتم بعد ذلك محاذاة ناقلات التمثيل للفصول المقابلة لتحقيق محاذاة علم الفئة للتمثيلات crosslingual. بالإضافة إلى ذلك، نقترح مواصلة مواءمة مقالات التمثيل لفئات الكلمات العالمية للألوج (أي أجزاء من العلاقات بين الكلام والاعتماد). على هذا النحو، يتم تقديم آلية تصفية جديدة لتسهيل تعلم تمثيلات فئة Word من تمثيلات السياق على نصوص الإدخال بناء على التعلم المشددي. نقوم بإجراء تجارب متشددة واسعة النطاق مع اللغة الإنجليزية والصينية والعربية على مهام ري. توضح النتائج فوائد الطريقة المقترحة التي تقدم بشكل كبير الأداء الحديث في هذه الإعدادات.
Previous work on crosslingual Relation and Event Extraction (REE) suffers from the monolingual bias issue due to the training of models on only the source language data. An approach to overcome this issue is to use unlabeled data in the target language to aid the alignment of crosslingual representations, i.e., via fooling a language discriminator. However, as this approach does not condition on class information, a target language example of a class could be incorrectly aligned to a source language example of a different class. To address this issue, we propose a novel crosslingual alignment method that leverages class information of REE tasks for representation learning. In particular, we propose to learn two versions of representation vectors for each class in an REE task based on either source or target language examples. Representation vectors for corresponding classes will then be aligned to achieve class-aware alignment for crosslingual representations. In addition, we propose to further align representation vectors for language-universal word categories (i.e., parts of speech and dependency relations). As such, a novel filtering mechanism is presented to facilitate the learning of word category representations from contextualized representations on input texts based on adversarial learning. We conduct extensive crosslingual experiments with English, Chinese, and Arabic over REE tasks. The results demonstrate the benefits of the proposed method that significantly advances the state-of-the-art performance in these settings.
References used
https://aclanthology.org/
rate research
Read More
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer. However, these multilingual encoders do not precisely align words and phrases across languages. Especially, le
arning alignments in the multilingual embedding space usually requires sentence-level or word-level parallel corpora, which are expensive to be obtained for low-resource languages. An alternative is to make the multilingual encoders more robust; when fine-tuning the encoder using downstream task, we train the encoder to tolerate noise in the contextual embedding spaces such that even if the representations of different languages are not aligned well, the model can still achieve good performance on zero-shot cross-lingual transfer. In this work, we propose a learning strategy for training robust models by drawing connections between adversarial examples and the failure cases of zero-shot cross-lingual transfer. We adopt two widely used robust training methods, adversarial training and randomized smoothing, to train the desired robust model. The experimental results demonstrate that robust training improves zero-shot cross-lingual transfer on text classification tasks. The improvement is more significant in the generalized cross-lingual transfer setting, where the pair of input sentences belong to two different languages.
Fully understanding narratives often requires identifying events in the context of whole documents and modeling the event relations. However, document-level event extraction is a challenging task as it requires the extraction of event and entity core
ference, and capturing arguments that span across different sentences. Existing works on event extraction usually confine on extracting events from single sentences, which fail to capture the relationships between the event mentions at the scale of a document, as well as the event arguments that appear in a different sentence than the event trigger. In this paper, we propose an end-to-end model leveraging Deep Value Networks (DVN), a structured prediction algorithm, to efficiently capture cross-event dependencies for document-level event extraction. Experimental results show that our approach achieves comparable performance to CRF-based models on ACE05, while enjoys significantly higher computational efficiency.
We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus (e.g. a few hundred sentence pairs). Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously transla
tes and reconstructs an input sentence. Through sharing model parameters among different languages, our model jointly trains the word embeddings in a common cross-lingual space. We also propose to combine word and subword embeddings to make use of orthographic similarities across different languages. We base our experiments on real-world data from endangered languages, namely Yongning Na, Shipibo-Konibo, and Griko. Our experiments on bilingual lexicon induction and word alignment tasks show that our model outperforms existing methods by a large margin for most language pairs. These results demonstrate that, contrary to common belief, an encoder-decoder translation model is beneficial for learning cross-lingual representations even in extremely low-resource conditions. Furthermore, our model also works well on high-resource conditions, achieving state-of-the-art performance on a German-English word-alignment task.
We address the task of automatic hate speech detection for low-resource languages. Rather than collecting and annotating new hate speech data, we show how to use cross-lingual transfer learning to leverage already existing data from higher-resource l
anguages. Using bilingual word embeddings based classifiers we achieve good performance on the target language by training only on the source dataset. Using our transferred system we bootstrap on unlabeled target language data, improving the performance of standard cross-lingual transfer approaches. We use English as a high resource language and German as the target language for which only a small amount of annotated corpora are available. Our results indicate that cross-lingual transfer learning together with our approach to leverage additional unlabeled data is an effective way of achieving good performance on low-resource target languages without the need for any target-language annotations.
Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings - maps of matching words across languages - without supervision. Despite these successes, GANs' performance for the difficult case of distant languages is
still not satisfactory. These limitations have been explained by GANs' incorrect assumption that source and target embedding spaces are related by a single linear mapping and are approximately isomorphic. We assume instead that, especially across distant languages, the mapping is only piece-wise linear, and propose a multi-adversarial learning method. This novel method induces the seed cross-lingual dictionary through multiple mappings, each induced to fit the mapping for one subspace. Our experiments on unsupervised bilingual lexicon induction and cross-lingual document classification show that this method improves performance over previous single-mapping methods, especially for distant languages.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
