Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userMultilingual Neural Machine Translation: Can Linguistic Hierarchies Help?
ترجمة متعددة اللغات الآلة العصبية: هل يمكن أن تساعد التسلسلات الهرمية اللغوية؟
370
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تقوم الترجمة العصبية متعددة اللغات (MNMT) بتدريب نموذج NMT واحد يدعم الترجمة بين لغات متعددة، بدلا من تدريب نماذج منفصلة لغات مختلفة. تعلم نموذج واحد يمكن أن يعزز الترجمة المنخفضة الموارد من خلال الاستفادة من البيانات من لغات متعددة. ومع ذلك، فإن أداء نموذج MNMT يعتمد اعتمادا كبيرا على نوع اللغات المستخدمة في التدريب، حيث أن نقل المعرفة من مجموعة متنوعة من اللغات تتحلل أداء الترجمة بسبب النقل السلبي. في هذه الورقة، نقترح مقاربة تقطير المعرفة التسلسل الهرمية (HKD) ل MNMT والتي تتمتع بالجماعات اللغوية التي تم إنشاؤها وفقا للميزات النموذجية والهلوجين من اللغات للتغلب على مسألة النقل السلبي. ينشئ HKD مجموعة من نماذج مساعد المعلم متعددة اللغات عبر آلية تقطير المعرفة الانتقائية تعتمد على مجموعات اللغات، ثم قم بالتقطير النموذج النهائي متعدد اللغات من المساعدين بطريقة تكيف. النتائج التجريبية المشتقة من مجموعة بيانات TED مع 53 لغة توضح فعالية نهجنا في تجنب تأثير النقل السلبي في MNMT، مما يؤدي إلى أداء ترجمة محسنة (حوالي 1 درجة بلو في المتوسط) مقارنة مع خطوط الأساس القوية.
Multilingual Neural Machine Translation (MNMT) trains a single NMT model that supports translation between multiple languages, rather than training separate models for different languages. Learning a single model can enhance the low-resource translation by leveraging data from multiple languages. However, the performance of an MNMT model is highly dependent on the type of languages used in training, as transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. In this paper, we propose a Hierarchical Knowledge Distillation (HKD) approach for MNMT which capitalises on language groups generated according to typological features and phylogeny of languages to overcome the issue of negative transfer. HKD generates a set of multilingual teacher-assistant models via a selective knowledge distillation mechanism based on the language groups, and then distills the ultimate multilingual model from those assistants in an adaptive way. Experimental results derived from the TED dataset with 53 languages demonstrate the effectiveness of our approach in avoiding the negative transfer effect in MNMT, leading to an improved translation performance (about 1 BLEU score in average) compared to strong baselines.
References used
https://aclanthology.org/
rate research
Read More
Multilingual Neural Machine Translation has achieved remarkable performance by training a single translation model for multiple languages. This paper describes our submission (Team ID: CFILT-IITB) for the MultiIndicMT: An Indic Language Multilingual
Task at WAT 2021. We train multilingual NMT systems by sharing encoder and decoder parameters with language embedding associated with each token in both encoder and decoder. Furthermore, we demonstrate the use of transliteration (script conversion) for Indic languages in reducing the lexical gap for training a multilingual NMT system. Further, we show improvement in performance by training a multilingual NMT system using languages of the same family, i.e., related languages.
Multilingual neural machine translation models typically handle one source language at a time. However, prior work has shown that translating from multiple source languages improves translation quality. Different from existing approaches on multi-sou
rce translation that are limited to the test scenario where parallel source sentences from multiple languages are available at inference time, we propose to improve multilingual translation in a more common scenario by exploiting synthetic source sentences from auxiliary languages. We train our model on synthetic multi-source corpora and apply random masking to enable flexible inference with single-source or bi-source inputs. Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that our model outperforms the multilingual baseline significantly by up to +4.0 BLEU with the largest improvements on low-resource or distant language pairs.
Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in
the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model's uncertainty on a small set of trusted clean data for multi-corpus machine translation. We experiments with two classes of uncertainty measures on multilingual (16 languages with 4 settings) and multi-domain settings (4 for in-domain and 2 for out-of-domain on English-German translation) and demonstrate our approach MultiUAT substantially outperforms its baselines, including both static and dynamic strategies. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.
Multimodal named entity recognition (MNER) requires to bridge the gap between language understanding and visual context. While many multimodal neural techniques have been proposed to incorporate images into the MNER task, the model's ability to lever
age multimodal interactions remains poorly understood. In this work, we conduct in-depth analyses of existing multimodal fusion techniques from different perspectives and describe the scenarios where adding information from the image does not always boost performance. We also study the use of captions as a way to enrich the context for MNER. Experiments on three datasets from popular social platforms expose the bottleneck of existing multimodal models and the situations where using captions is beneficial.
This report describes Microsoft's machine translation systems for the WMT21 shared task on large-scale multilingual machine translation. We participated in all three evaluation tracks including Large Track and two Small Tracks where the former one is
unconstrained and the latter two are fully constrained. Our model submissions to the shared task were initialized with DeltaLM, a generic pre-trained multilingual encoder-decoder model, and fine-tuned correspondingly with the vast collected parallel data and allowed data sources according to track settings, together with applying progressive learning and iterative back-translation approaches to further improve the performance. Our final submissions ranked first on three tracks in terms of the automatic evaluation metric.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
