Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userExploiting Curriculum Learning in Unsupervised Neural Machine Translation
استغلال منهج التعلم في الترجمة الآلية العصبية غير المنشأة
381
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
أصبحت الترجمة المرجودة (BT) واحدة من مكونات الأمر الواقع في الترجمة الآلية العصبية غير المنشأة (UNMT)، ويجعل صراحة لديها القدرة على الترجمة. ومع ذلك، يتم التعامل مع جميع النصوص الثنائية الزائفة التي تم إنشاؤها بواسطة BT بنفس القدر كبيانات نظيفة أثناء التحسين دون النظر في تنوع الجودة، مما يؤدي إلى التقارب البطيء وأداء الترجمة المحدود. لمعالجة هذه المشكلة، نقترح طريقة تعلم المناهج الدراسية للاستفادة تدريجيا من النصوص الثنائية الزائفة القائمة على جودتها من التعبيات المتعددة. على وجه التحديد، نقوم أولا بتطبيق تضمين كلمة crosslingual لحساب صعوبة الترجمة المحتملة (الجودة) للجمل الأولية. بعد ذلك، يتم تغذية الجمل في برنامج التعريف الخاص ب UNMT من السهل إلى الدفعة الصلبة عن طريق الدفعة. علاوة على ذلك، بالنظر إلى جودة الجمل / الرموز في دفعة معينة هي متنوعة أيضا، فإننا نتخذ النموذج نفسه لحساب درجات الجودة المحبوبة بشكل جيد، والتي يتم تقديمها كعامل تعليمي لموازنة مساهمات أجزاء مختلفة عند فقد الحوسبة وتشجيعها نموذج UNMT للتركيز على البيانات الزائفة بجودة أعلى. النتائج التجريبية على WMT 14 EN-FR، WMT 14 EN-DE، WMT 16 EN-RO، و LDC EN-ZH توضح أن الطريقة المقترحة تحقق تحسينات ثابتة مع سرعة التقارب الأسرع.
Back-translation (BT) has become one of the de facto components in unsupervised neural machine translation (UNMT), and it explicitly makes UNMT have translation ability. However, all the pseudo bi-texts generated by BT are treated equally as clean data during optimization without considering the quality diversity, leading to slow convergence and limited translation performance. To address this problem, we propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. Specifically, we first apply crosslingual word embedding to calculate the potential translation difficulty (quality) for the monolingual sentences. Then, the sentences are fed into UNMT from easy to hard batch by batch. Furthermore, considering the quality of sentences/tokens in a particular batch are also diverse, we further adopt the model itself to calculate the fine-grained quality scores, which are served as learning factors to balance the contributions of different parts when computing loss and encourage the UNMT model to focus on pseudo data with higher quality. Experimental results on WMT 14 En-Fr, WMT 14 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks demonstrate that the proposed method achieves consistent improvements with faster convergence speed.
References used
https://aclanthology.org/
rate research
Read More
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of easy'' samples from training data at the early training stage. This is not always achievable for low-resource languages where the amoun
t of training data is limited. To address such a limitation, we propose a novel token-wise curriculum learning approach that creates sufficient amounts of easy samples. Specifically, the model learns to predict a short sub-sequence from the beginning part of each target sentence at the early stage of training. Then the sub-sequence is gradually expanded as the training progresses. Such a new curriculum design is inspired by the cumulative effect of translation errors, which makes the latter tokens more challenging to predict than the beginning ones. Extensive experiments show that our approach can consistently outperform baselines on five language pairs, especially for low-resource languages. Combining our approach with sentence-level methods further improves the performance of high-resource languages.
Unsupervised neural machine translation (UNMT) that relies solely on massive monolingual corpora has achieved remarkable results in several translation tasks. However, in real-world scenarios, massive monolingual corpora do not exist for some extreme
ly low-resource languages such as Estonian, and UNMT systems usually perform poorly when there is not adequate training corpus for one language. In this paper, we first define and analyze the unbalanced training data scenario for UNMT. Based on this scenario, we propose UNMT self-training mechanisms to train a robust UNMT system and improve its performance in this case. Experimental results on several language pairs show that the proposed methods substantially outperform conventional UNMT systems.
Machine translation usually relies on parallel corpora to provide parallel signals for training. The advent of unsupervised machine translation has brought machine translation away from this reliance, though performance still lags behind traditional
supervised machine translation. In unsupervised machine translation, the model seeks symmetric language similarities as a source of weak parallel signal to achieve translation. Chomsky's Universal Grammar theory postulates that grammar is an innate form of knowledge to humans and is governed by universal principles and constraints. Therefore, in this paper, we seek to leverage such shared grammar clues to provide more explicit language parallel signals to enhance the training of unsupervised machine translation models. Through experiments on multiple typical language pairs, we demonstrate the effectiveness of our proposed approaches.
Interactive-predictive translation is a collaborative iterative process and where human translators produce translations with the help of machine translation (MT) systems interactively. Various sampling techniques in active learning (AL) exist to upd
ate the neural MT (NMT) model in the interactive-predictive scenario. In this paper and we explore term based (named entity count (NEC)) and quality based (quality estimation (QE) and sentence similarity (Sim)) sampling techniques -- which are used to find the ideal candidates from the incoming data -- for human supervision and MT model's weight updation. We carried out experiments with three language pairs and viz. German-English and Spanish-English and Hindi-English. Our proposed sampling technique yields 1.82 and 0.77 and 0.81 BLEU points improvements for German-English and Spanish-English and Hindi-English and respectively and over random sampling based baseline. It also improves the present state-of-the-art by 0.35 and 0.12 BLEU points for German-English and Spanish-English and respectively. Human editing effort in terms of number-of-words-changed also improves by 5 and 4 points for German-English and Spanish-English and respectively and compared to the state-of-the-art.
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper and we propose two simple search base
d curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally and we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system using contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
