Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userImproving Multilingual Translation by Representation and Gradient Regularization
تحسين الترجمة متعددة اللغات عن طريق التنظيم والتدرج
278
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تمكن الترجمة الآلية العصبية متعددة اللغات (NMT) نموذج واحد لخدمة جميع اتجاهات الترجمة، بما في ذلك تلك التي هي غير مرئية أثناء التدريب، I.E. Zero-Shot الترجمة. على الرغم من أن النماذج الحالية جذابة من الناحية النظرية غالبا ما تنتج ترجمات منخفضة الجودة - لا تفشل عادة في إنتاج مخرجات باللغة المستهدفة الصحيحة. في هذا العمل، نلاحظ أن الترجمة المستهلكة المستهدفة هي المهيمنة حتى في أنظمة قوية متعددة اللغات، تدربت على كورسا متعددة اللغات الضخمة. لمعالجة هذه المشكلة، نقترح نهج مشترك لتنظيم نماذج NMT على مستوى التمثيل ومستوى التدرج. في مستوى التمثيل، نستفيد مهمة التنبؤ باللغة المستهدفة المساعدة لتنظيم مخرجات فك ترميز الكفر للاحتفاظ بمعلومات حول اللغة المستهدفة. عند مستوى التدرج، نستفيد كمية صغيرة من البيانات المباشرة (بآلاف أزواج الجملة) لتنظيم تدرجات النماذج. توضح نتائجنا أن نهجنا فعال للغاية في حد سواء تقليل حوادث الترجمة المستهدفة وتحسين أداء الترجمة الصفرية بواسطة +5.59 و +10.38 بلو على مجموعات بيانات WMT و OPUS على التوالي. علاوة على ذلك، تظهر التجارب أن طريقتنا تعمل أيضا بشكل جيد عندما لا يتوفر كمية صغيرة من البيانات المباشرة.
Multilingual Neural Machine Translation (NMT) enables one model to serve all translation directions, including ones that are unseen during training, i.e. zero-shot translation. Despite being theoretically attractive, current models often produce low quality translations -- commonly failing to even produce outputs in the right target language. In this work, we observe that off-target translation is dominant even in strong multilingual systems, trained on massive multilingual corpora. To address this issue, we propose a joint approach to regularize NMT models at both representation-level and gradient-level. At the representation level, we leverage an auxiliary target language prediction task to regularize decoder outputs to retain information about the target language. At the gradient level, we leverage a small amount of direct data (in thousands of sentence pairs) to regularize model gradients. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance by +5.59 and +10.38 BLEU on WMT and OPUS datasets respectively. Moreover, experiments show that our method also works well when the small amount of direct data is not available.
References used
https://aclanthology.org/
rate research
Read More
Multilingual neural machine translation models typically handle one source language at a time. However, prior work has shown that translating from multiple source languages improves translation quality. Different from existing approaches on multi-sou
rce translation that are limited to the test scenario where parallel source sentences from multiple languages are available at inference time, we propose to improve multilingual translation in a more common scenario by exploiting synthetic source sentences from auxiliary languages. We train our model on synthetic multi-source corpora and apply random masking to enable flexible inference with single-source or bi-source inputs. Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that our model outperforms the multilingual baseline significantly by up to +4.0 BLEU with the largest improvements on low-resource or distant language pairs.
Developing a unified multilingual model has been a long pursuing goal for machine translation. However, existing approaches suffer from performance degradation - a single multilingual model is inferior to separately trained bilingual ones on rich-res
ource languages. We conjecture that such a phenomenon is due to interference brought by joint training with multiple languages. To accommodate the issue, we propose CIAT, an adapted Transformer model with a small parameter overhead for multilingual machine translation. We evaluate CIAT on multiple benchmark datasets, including IWSLT, OPUS-100, and WMT. Experiments show that the CIAT consistently outperforms strong multilingual baselines on 64 of total 66 language directions, 42 of which have above 0.5 BLEU improvement.
Can pre-trained BERT for one language and GPT for another be glued together to translate texts? Self-supervised training using only monolingual data has led to the success of pre-trained (masked) language models in many NLP tasks. However, directly c
onnecting BERT as an encoder and GPT as a decoder can be challenging in machine translation, for GPT-like models lack a cross-attention component that is needed in seq2seq decoders. In this paper, we propose Graformer to graft separately pre-trained (masked) language models for machine translation. With monolingual data for pre-training and parallel data for grafting training, we maximally take advantage of the usage of both types of data. Experiments on 60 directions show that our method achieves average improvements of 5.8 BLEU in x2en and 2.9 BLEU in en2x directions comparing with the multilingual Transformer of the same size.
Neural machine translation based on bilingual text with limited training data suffers from lexical diversity, which lowers the rare word translation accuracy and reduces the generalizability of the translation system. In this work, we utilise the mul
tiple captions from the Multi-30K dataset to increase the lexical diversity aided with the cross-lingual transfer of information among the languages in a multilingual setup. In this multilingual and multimodal setting, the inclusion of the visual features boosts the translation quality by a significant margin. Empirical study affirms that our proposed multimodal approach achieves substantial gain in terms of the automatic score and shows robustness in handling the rare word translation in the pretext of English to/from Hindi and Telugu translation tasks.
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper and we propose two simple search base
d curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally and we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system using contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
