Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training
عدم اليقين - موازنة لتدريب ترجمة الآلات العصبية متعددة اللغات ومتعددة اللغات
336
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعلم نموذج الترجمة متعددة اللغات ومتعدد اللغات يمثل تحديا لأن البيانات غير المتجانسة والمخطورة تجعل النموذج تتلاقص بشكل غير متسق على مختلف كوربورا في العالم الحقيقي. تتمثل هذه الممارسة الشائعة في ضبط حصة كل جثة في التدريب، بحيث يمكن أن تستفيد عملية التعلم الحالات المتوازنة والموارد المنخفضة من الموارد العالية. ومع ذلك، عادة ما تعتمد أساليب موازنة التلقائي عادة على الخصائص داخل ومشتركة بين البيانات، والتي عادة ما تكون غير مرغقة أو تتطلب من الشاورات البشرية. في هذا العمل، نقترح نهجا، مواد متعددة، أن ضبط استخدام بيانات التدريب بشكل حيوي استنادا إلى عدم اليقين في النموذج على مجموعة صغيرة من البيانات النظيفة الموثوقة للترجمة متعددة الكائنات. نحن تجارب مع فئتين من تدابير عدم اليقين في تعدد اللغات (16 لغة مع 4 إعدادات) وإعدادات متعددة النجانات (4 للمجال في المجال و 2 للخارج على الترجمة الإنجليزية-الألمانية) وإظهار نهجنا متعدد الاستخدامات بشكل كبير خطوط الأساس، بما في ذلك الاستراتيجيات الثابتة والديناميكية. نقوم بتحليل النقل عبر المجال وإظهار نقص الأساليب القائمة على الاستقرار والمشاكل.
Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model's uncertainty on a small set of trusted clean data for multi-corpus machine translation. We experiments with two classes of uncertainty measures on multilingual (16 languages with 4 settings) and multi-domain settings (4 for in-domain and 2 for out-of-domain on English-German translation) and demonstrate our approach MultiUAT substantially outperforms its baselines, including both static and dynamic strategies. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.
References used
https://aclanthology.org/
rate research
Read More
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper and we propose two simple search base
d curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally and we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system using contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.
Multilingual neural machine translation models typically handle one source language at a time. However, prior work has shown that translating from multiple source languages improves translation quality. Different from existing approaches on multi-sou
rce translation that are limited to the test scenario where parallel source sentences from multiple languages are available at inference time, we propose to improve multilingual translation in a more common scenario by exploiting synthetic source sentences from auxiliary languages. We train our model on synthetic multi-source corpora and apply random masking to enable flexible inference with single-source or bi-source inputs. Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that our model outperforms the multilingual baseline significantly by up to +4.0 BLEU with the largest improvements on low-resource or distant language pairs.
Neural machine translation based on bilingual text with limited training data suffers from lexical diversity, which lowers the rare word translation accuracy and reduces the generalizability of the translation system. In this work, we utilise the mul
tiple captions from the Multi-30K dataset to increase the lexical diversity aided with the cross-lingual transfer of information among the languages in a multilingual setup. In this multilingual and multimodal setting, the inclusion of the visual features boosts the translation quality by a significant margin. Empirical study affirms that our proposed multimodal approach achieves substantial gain in terms of the automatic score and shows robustness in handling the rare word translation in the pretext of English to/from Hindi and Telugu translation tasks.
Multilingual Neural Machine Translation (MNMT) trains a single NMT model that supports translation between multiple languages, rather than training separate models for different languages. Learning a single model can enhance the low-resource translat
ion by leveraging data from multiple languages. However, the performance of an MNMT model is highly dependent on the type of languages used in training, as transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. In this paper, we propose a Hierarchical Knowledge Distillation (HKD) approach for MNMT which capitalises on language groups generated according to typological features and phylogeny of languages to overcome the issue of negative transfer. HKD generates a set of multilingual teacher-assistant models via a selective knowledge distillation mechanism based on the language groups, and then distills the ultimate multilingual model from those assistants in an adaptive way. Experimental results derived from the TED dataset with 53 languages demonstrate the effectiveness of our approach in avoiding the negative transfer effect in MNMT, leading to an improved translation performance (about 1 BLEU score in average) compared to strong baselines.
Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models. However, the heavy data imbalance between languages
hinders the model from performing uniformly across language pairs. In this paper, we propose a new learning objective for MNMT based on distributionally robust optimization, which minimizes the worst-case expected loss over the set of language pairs. We further show how to practically optimize this objective for large translation corpora using an iterated best response scheme, which is both effective and incurs negligible additional computational cost compared to standard empirical risk minimization. We perform extensive experiments on three sets of languages from two datasets and show that our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
