Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCompetence-based Curriculum Learning for Multilingual Machine Translation
التعلم مناهج الكفاءة القائمة على الترجمة متعددة اللغات
207
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
حاليا، تتلقى الترجمة متعددة اللغات الآلية أكثر اهتماما أكثر وأكثر لأنها تجلب أداء أفضل لغات الموارد المنخفضة (LRLS) وتوفر مساحة أكبر. ومع ذلك، فإن نماذج الترجمة متعددة اللغات الحالية تواجه تحديا شديدا: عدم التوازن. نتيجة لذلك، فإن أداء الترجمة من لغات مختلفة في نماذج الترجمة متعددة اللغات مختلفة تماما. نقول أن مشكلة الاختلال هذه تنبع من كفاءات التعليم المختلفة لغات مختلفة. لذلك، نحن نركز على تحقيق التوازن بين الكفاءات التعليمية لغات مختلفة واقتراح مناهج التعلم القائم على الكفاءة للترجمة الآلية متعددة اللغات، والتي تسمى CCL-M. على وجه التحديد، نقوم أولا بتحديد كفاءتين للمساعدة في جدولة لغات الموارد العالية (HRLS) ولغات المورد المنخفضة: 1) الكفاءة التي تم تقييمها ذاتيا، وتقييم مدى تعلم اللغة نفسها؛ 2) الكفاءة التي تم تقييمها HRLS، وتقييم ما إذا كانت LRL جاهزة للتعلم وفقا لخلاف HRLS الذي تم تقييمه الذاتي. استنادا إلى الكفاءات المذكورة أعلاه، نستخدم خوارزمية CCL-M المقترحة إضافة لغات جديدة تدريجيا في التدريب المحدد بطريقة تعلم المناهج الدراسية. علاوة على ذلك، نقترح استراتيجية أخذان رصاصة ديناميكية متوازنة من الكفاءة النووية لتحسين عينات التدريب بشكل أفضل في التدريب متعدد اللغات. تظهر النتائج التجريبية أن نهجنا حقق مكاسب أداء ثابتة وهامة مقارنة بالنهج السابق للدولة السابقة بشأن مجموعة بيانات محادثات تيد.
Currently, multilingual machine translation is receiving more and more attention since it brings better performance for low resource languages (LRLs) and saves more space. However, existing multilingual machine translation models face a severe challenge: imbalance. As a result, the translation performance of different languages in multilingual translation models are quite different. We argue that this imbalance problem stems from the different learning competencies of different languages. Therefore, we focus on balancing the learning competencies of different languages and propose Competence-based Curriculum Learning for Multilingual Machine Translation, named CCL-M. Specifically, we firstly define two competencies to help schedule the high resource languages (HRLs) and the low resource languages: 1) Self-evaluated Competence, evaluating how well the language itself has been learned; and 2) HRLs-evaluated Competence, evaluating whether an LRL is ready to be learned according to HRLs' Self-evaluated Competence. Based on the above competencies, we utilize the proposed CCL-M algorithm to gradually add new languages into the training set in a curriculum learning manner. Furthermore, we propose a novel competence-aware dynamic balancing sampling strategy for better selecting training samples in multilingual training. Experimental results show that our approach has achieved a steady and significant performance gain compared to the previous state-of-the-art approach on the TED talks dataset.
References used
https://aclanthology.org/
rate research
Read More
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper and we propose two simple search base
d curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally and we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system using contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
Recent works have found evidence of gender bias in models of machine translation and coreference resolution using mostly synthetic diagnostic datasets. While these quantify bias in a controlled experiment, they often do so on a small scale and consis
t mostly of artificial, out-of-distribution sentences. In this work, we find grammatical patterns indicating stereotypical and non-stereotypical gender-role assignments (e.g., female nurses versus male dancers) in corpora from three domains, resulting in a first large-scale gender bias dataset of 108K diverse real-world English sentences. We manually verify the quality of our corpus and use it to evaluate gender bias in various coreference resolution and machine translation models. We find that all tested models tend to over-rely on gender stereotypes when presented with natural inputs, which may be especially harmful when deployed in commercial systems. Finally, we show that our dataset lends itself to finetuning a coreference resolution model, finding it mitigates bias on a held out set. Our dataset and models are publicly available at github.com/SLAB-NLP/BUG. We hope they will spur future research into gender bias evaluation mitigation techniques in realistic settings.
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of easy'' samples from training data at the early training stage. This is not always achievable for low-resource languages where the amoun
t of training data is limited. To address such a limitation, we propose a novel token-wise curriculum learning approach that creates sufficient amounts of easy samples. Specifically, the model learns to predict a short sub-sequence from the beginning part of each target sentence at the early stage of training. Then the sub-sequence is gradually expanded as the training progresses. Such a new curriculum design is inspired by the cumulative effect of translation errors, which makes the latter tokens more challenging to predict than the beginning ones. Extensive experiments show that our approach can consistently outperform baselines on five language pairs, especially for low-resource languages. Combining our approach with sentence-level methods further improves the performance of high-resource languages.
Developing a unified multilingual model has been a long pursuing goal for machine translation. However, existing approaches suffer from performance degradation - a single multilingual model is inferior to separately trained bilingual ones on rich-res
ource languages. We conjecture that such a phenomenon is due to interference brought by joint training with multiple languages. To accommodate the issue, we propose CIAT, an adapted Transformer model with a small parameter overhead for multilingual machine translation. We evaluate CIAT on multiple benchmark datasets, including IWSLT, OPUS-100, and WMT. Experiments show that the CIAT consistently outperforms strong multilingual baselines on 64 of total 66 language directions, 42 of which have above 0.5 BLEU improvement.
We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation. Our approach is suitable f
or large-scale datasets, applies to distant languages with unseen scripts, incurs only minor degradation on the translation performance for the original language pairs and provides competitive performance even in the case where we only possess monolingual data for the new languages.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
