Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userHarnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
تسخير تعددا متعددة اللغات في الترجمة الآلية غير المنشورة لغات نادرة
315
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
وصلت الترجمة غير المزدئة إلى أداء مثير للإعجاب على أزواج اللغة الغنية بالموارد مثل اللغة الإنجليزية الفرنسية والإنجليزية - الألمانية. ومع ذلك، أظهرت الدراسات المبكرة أنه في بيئات أكثر واقعية تنطوي على الموارد المنخفضة، لغات نادرة، تؤدي الترجمة غير المعينة بشكل سيئ، وتحقيق أقل من 3.0 بلو. في هذا العمل، نظير على أن تعددية اللغات أمر بالغ الأهمية لجعل أنظمة غير مخالفة عملية لإعدادات الموارد المنخفضة. على وجه الخصوص، نقدم نموذجا واحدا ل 5 لغات منخفضة الموارد (الغوجاراتية، كازاخ، النيبالية، السنهالية، والتركية) من وإلى اتجاهات اللغة الإنجليزية، والتي ترفع البيانات المتوازية غير المباشرة والمساعدة من أزواج لغة موارد أخرى عالية من خلال ثلاثة مخطط التدريب المرحلة. نحن نتفوق على جميع خطوط الأساس غير المدعومة الحالية له لهذه اللغات، وتحقيق مكاسب تصل إلى 14.4 بلو. بالإضافة إلى ذلك، نحن نتفوق خطوط خطوط خطوط خطوط خطوط أخرى تحت إشراف قوية لمختلف أزواج اللغات وكذلك تطابق أداء النموذج الحالي للإشراف على النيبالية والإنجليزية. نقوم بإجراء سلسلة من دراسات الاجتثاث لإثبات نطاقات نموذجنا بموجب درجات مختلفة من جودة البيانات، وكذلك لتحليل العوامل التي أدت إلى الأداء الفائق للنهج المقترح على النماذج التقليدية غير المعروضة.
Unsupervised translation has reached impressive performance on resource-rich language pairs such as English-French and English-German. However, early studies have shown that in more realistic settings involving low-resource, rare languages, unsupervised translation performs poorly, achieving less than 3.0 BLEU. In this work, we show that multilinguality is critical to making unsupervised systems practical for low-resource settings. In particular, we present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali, Sinhala, and Turkish) to and from English directions, which leverages monolingual and auxiliary parallel data from other high-resource language pairs via a three-stage training scheme. We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU. Additionally, we outperform strong supervised baselines for various language pairs as well as match the performance of the current state-of-the-art supervised model for Nepali-English. We conduct a series of ablation studies to establish the robustness of our model under different degrees of data quality, as well as to analyze the factors which led to the superior performance of the proposed approach over traditional unsupervised models.
References used
https://aclanthology.org/
rate research
Read More
For most language combinations and parallel data is either scarce or simply unavailable. To address this and unsupervised machine translation (UMT) exploits large amounts of monolingual data by using synthetic data generation techniques such as back-
translation and noising and while self-supervised NMT (SSNMT) identifies parallel sentences in smaller comparable data and trains on them. To this date and the inclusion of UMT data generation techniques in SSNMT has not been investigated. We show that including UMT techniques into SSNMT significantly outperforms SSNMT (up to +4.3 BLEU and af2en) as well as statistical (+50.8 BLEU) and hybrid UMT (+51.5 BLEU) baselines on related and distantly-related and unrelated language pairs.
Dravidian languages, such as Kannada and Tamil, are notoriously difficult to translate by state-of-the-art neural models. This stems from the fact that these languages are morphologically very rich as well as being low-resourced. In this paper, we fo
cus on subword segmentation and evaluate Linguistically Motivated Vocabulary Reduction (LMVR) against the more commonly used SentencePiece (SP) for the task of translating from English into four different Dravidian languages. Additionally we investigate the optimal subword vocabulary size for each language. We find that SP is the overall best choice for segmentation, and that larger dictionary sizes lead to higher translation quality.
Back-translation (BT) has become one of the de facto components in unsupervised neural machine translation (UNMT), and it explicitly makes UNMT have translation ability. However, all the pseudo bi-texts generated by BT are treated equally as clean da
ta during optimization without considering the quality diversity, leading to slow convergence and limited translation performance. To address this problem, we propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. Specifically, we first apply crosslingual word embedding to calculate the potential translation difficulty (quality) for the monolingual sentences. Then, the sentences are fed into UNMT from easy to hard batch by batch. Furthermore, considering the quality of sentences/tokens in a particular batch are also diverse, we further adopt the model itself to calculate the fine-grained quality scores, which are served as learning factors to balance the contributions of different parts when computing loss and encourage the UNMT model to focus on pseudo data with higher quality. Experimental results on WMT 14 En-Fr, WMT 14 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks demonstrate that the proposed method achieves consistent improvements with faster convergence speed.
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper and we propose two simple search base
d curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally and we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system using contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
Multilingual neural machine translation models typically handle one source language at a time. However, prior work has shown that translating from multiple source languages improves translation quality. Different from existing approaches on multi-sou
rce translation that are limited to the test scenario where parallel source sentences from multiple languages are available at inference time, we propose to improve multilingual translation in a more common scenario by exploiting synthetic source sentences from auxiliary languages. We train our model on synthetic multi-source corpora and apply random masking to enable flexible inference with single-source or bi-source inputs. Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that our model outperforms the multilingual baseline significantly by up to +4.0 BLEU with the largest improvements on low-resource or distant language pairs.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
