Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples
AM2ICO: تقييم معنى الكلمات في السياق عبر لغات الموارد المنخفضة مع أمثلة الخصومة
425
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
إن التقاط معنى كلمة في السياق والتمييز بين المراسلات والاختلافات عبر اللغات هو مفتاح بناء نماذج تمثيل نصية متعددة اللغات والنجاح. ومع ذلك، فإن مجموعات بيانات التقييم المتعددة اللغات الحالية التي تقيم الدلالات المعجمية في السياق "لها قيود مختلفة. على وجه الخصوص، 1) تقتصر تغطيتها اللغوية على لغات الموارد العالية والانحساج لصالح سوى عدد قليل من الأسر والمناطق اللغوية، 2) تصميم يجعل المهمة قابلة للحل عبر الإشارات السطحية، والتي تؤدي إلى تضخم بشكل مصطنع (وأحيانا سوبر -Human) عروض التشفير المحددين، و 3) لا يوجد دعم للتقييم المتبادل اللغوي. من أجل معالجة هذه الثغرات، نقدم AM2ICO (المعنى الخصومة والتعدد اللغات في السياق)، مجموعة التقييم الواسع التغطية واللغات متعددة اللغات؛ يهدف إلى تقييم إدراج قدرة نماذج التمثيل الحديثة (SOTA) لفهم هوية معنى الكلمات في السياقات المتبادلة لمدة 14 أزواج لغوية. نقوم بإجراء سلسلة من التجارب في مجموعة واسعة من الإعدادات وإظهار الطبيعة الصعبة ل AM2ICO. تكشف النتائج أن تشفير سوتا الحالية التي يتعامل معها يتخلف بشكل كبير وراء الأداء البشري، ويتم ملاحظة أكبر فجوات لغات الموارد المنخفضة ولغات مختلفة عن اللغة الإنجليزية.
Capturing word meaning in context and distinguishing between correspondences and variations across languages is key to building successful multilingual and cross-lingual text representation models. However, existing multilingual evaluation datasets that evaluate lexical semantics in-context'' have various limitations. In particular, 1) their language coverage is restricted to high-resource languages and skewed in favor of only a few language families and areas, 2) a design that makes the task solvable via superficial cues, which results in artificially inflated (and sometimes super-human) performances of pretrained encoders, and 3) no support for cross-lingual evaluation. In order to address these gaps, we present AM2iCo (Adversarial and Multilingual Meaning in Context), a wide-coverage cross-lingual and multilingual evaluation set; it aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts for 14 language pairs. We conduct a series of experiments in a wide range of setups and demonstrate the challenging nature of AM2iCo. The results reveal that current SotA pretrained encoders substantially lag behind human performance, and the largest gaps are observed for low-resource languages and languages dissimilar to English.
References used
https://aclanthology.org/
rate research
Read More
Cross-lingual word embeddings provide a way for information to be transferred between languages. In this paper we evaluate an extension of a joint training approach to learning cross-lingual embeddings that incorporates sub-word information during tr
aining. This method could be particularly well-suited to lower-resource and morphologically-rich languages because it can be trained on modest size monolingual corpora, and is able to represent out-of-vocabulary words (OOVs). We consider bilingual lexicon induction, including an evaluation focused on OOVs. We find that this method achieves improvements over previous approaches, particularly for OOVs.
We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus (e.g. a few hundred sentence pairs). Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously transla
tes and reconstructs an input sentence. Through sharing model parameters among different languages, our model jointly trains the word embeddings in a common cross-lingual space. We also propose to combine word and subword embeddings to make use of orthographic similarities across different languages. We base our experiments on real-world data from endangered languages, namely Yongning Na, Shipibo-Konibo, and Griko. Our experiments on bilingual lexicon induction and word alignment tasks show that our model outperforms existing methods by a large margin for most language pairs. These results demonstrate that, contrary to common belief, an encoder-decoder translation model is beneficial for learning cross-lingual representations even in extremely low-resource conditions. Furthermore, our model also works well on high-resource conditions, achieving state-of-the-art performance on a German-English word-alignment task.
In this work, we investigate methods for the challenging task of translating between low- resource language pairs that exhibit some level of similarity. In particular, we consider the utility of transfer learning for translating between several Indo-
European low-resource languages from the Germanic and Romance language families. In particular, we build two main classes of transfer-based systems to study how relatedness can benefit the translation performance. The primary system fine-tunes a model pre-trained on a related language pair and the contrastive system fine-tunes one pre-trained on an unrelated language pair. Our experiments show that although relatedness is not necessary for transfer learning to work, it does benefit model performance.
The widespread presence of offensive language on social media motivated the development of systems capable of recognizing such content automatically. Apart from a few notable exceptions, most research on automatic offensive language identification ha
s dealt with English. To address this shortcoming, we introduce MOLD, the Marathi Offensive Language Dataset. MOLD is the first dataset of its kind compiled for Marathi, thus opening a new domain for research in low-resource Indo-Aryan languages. We present results from several machine learning experiments on this dataset, including zero-short and other transfer learning experiments on state-of-the-art cross-lingual transformers from existing data in Bengali, English, and Hindi.
Cross-lingual word embeddings (CLWEs) have proven indispensable for various natural language processing tasks, e.g., bilingual lexicon induction (BLI). However, the lack of data often impairs the quality of representations. Various approaches requiri
ng only weak cross-lingual supervision were proposed, but current methods still fail to learn good CLWEs for languages with only a small monolingual corpus. We therefore claim that it is necessary to explore further datasets to improve CLWEs in low-resource setups. In this paper we propose to incorporate data of related high-resource languages. In contrast to previous approaches which leverage independently pre-trained embeddings of languages, we (i) train CLWEs for the low-resource and a related language jointly and (ii) map them to the target language to build the final multilingual space. In our experiments we focus on Occitan, a low-resource Romance language which is often neglected due to lack of resources. We leverage data from French, Spanish and Catalan for training and evaluate on the Occitan-English BLI task. By incorporating supporting languages our method outperforms previous approaches by a large margin. Furthermore, our analysis shows that the degree of relatedness between an incorporated language and the low-resource language is critically important.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
