Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages
Afromt: استراتيجيات الاحتجاج والمعايير القابلة للتكرار للترجمة 8 اللغات الأفريقية
102
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
المعايير القابلة للتكرار حاسمة في قيادة التقدم المحرز في أبحاث الترجمة الآلية. ومع ذلك، تقتصر معايير الترجمة الآلية الموجودة في الغالب على اللغات عالية الموارد أو ممثلة تمثيلا جيدا. على الرغم من الاهتمام المتزايد في الترجمة ذات الجهاز المنخفض، لا توجد معايير موحدة استنساخ للعديد من اللغات الأفريقية، يتم استخدام العديد منها من قبل ملايين المتحدثين ولكن لديهم بيانات نصية أقل رقمية. لمعالجة هذه التحديات، نقترح AFROMT، معيارا قياسيا موحدا ونينا ونظيفا ولاكاثر لثمانية لغات أفريقية منطوقة على نطاق واسع. نقوم أيضا بتطوير مجموعة من أدوات التحليل لتشخيص النظام مع الأخذ في الاعتبار الخصائص الفريدة لهذه اللغات. علاوة على ذلك، نستكشف القضية التي تعتبرها مؤلمة تركز على الموارد المنخفضة المنخفضة وتطوير استراتيجيات جديدة قائمة على زيادة تكبير البيانات، واستفادة من معلومات محاذاة مستوى الكلمات والبيانات الزائفة غير المباشرة لإحاطاء نماذج التسلسل متعددة اللغات. نوضح تحسينات كبيرة عند الاحتجاج في 11 لغة، مع مكاسب تصل إلى 2 نقطة بلو على خطوط خطوط خطوط خطوط طويلة. نعرض أيضا مكاسب تصل إلى 12 نقطة بلو على خطوط خطوط نقل النقل عبر اللغات في سيناريوهات مقيدة البيانات. سيتم إصدار جميع النماذج من التعليمات البرمجية والأعمدة كخطوات أخرى نحو معايير قابلة لإزالة الأكبرات للغات الأفريقية.
Reproducible benchmarks are crucial in driving progress of machine translation research. However, existing machine translation benchmarks have been mostly limited to high-resource or well-represented languages. Despite an increasing interest in low-resource machine translation, there are no standardized reproducible benchmarks for many African languages, many of which are used by millions of speakers but have less digitized textual data. To tackle these challenges, we propose AfroMT, a standardized, clean, and reproducible machine translation benchmark for eight widely spoken African languages. We also develop a suite of analysis tools for system diagnosis taking into account the unique properties of these languages. Furthermore, we explore the newly considered case of low-resource focused pretraining and develop two novel data augmentation-based strategies, leveraging word-level alignment information and pseudo-monolingual data for pretraining multilingual sequence-to-sequence models. We demonstrate significant improvements when pretraining on 11 languages, with gains of up to 2 BLEU points over strong baselines. We also show gains of up to 12 BLEU points over cross-lingual transfer baselines in data-constrained scenarios. All code and pretrained models will be released as further steps towards larger reproducible benchmarks for African languages.
References used
https://aclanthology.org/
rate research
Read More
Abstract We take a step towards addressing the under- representation of the African continent in NLP research by bringing together different stakeholders to create the first large, publicly available, high-quality dataset for named entity recognition
(NER) in ten African languages. We detail the characteristics of these languages to help researchers and practitioners better understand the challenges they pose for NER tasks. We analyze our datasets and conduct an extensive empirical evaluation of state- of-the-art methods across both supervised and transfer learning settings. Finally, we release the data, code, and models to inspire future research on African NLP.1
This paper describes a methodology for syntactic knowledge transfer between high-resource languages to extremely low-resource languages. The methodology consists in leveraging multilingual BERT self-attention model pretrained on large datasets to dev
elop a multilingual multi-task model that can predict Universal Dependencies annotations for three African low-resource languages. The UD annotations include universal part-of-speech, morphological features, lemmas, and dependency trees. In our experiments, we used multilingual word embeddings and a total of 11 Universal Dependencies treebanks drawn from three high-resource languages (English, French, Norwegian) and three low-resource languages (Bambara, Wolof and Yoruba). We developed various models to test specific language combinations involving contemporary contact languages or genetically related languages. The results of the experiments show that multilingual models that involve high-resource languages and low-resource languages with contemporary contact between each other can provide better results than combinations that only include unrelated languages. As far genetic relationships are concerned, we could not draw any conclusion regarding the impact of language combinations involving the selected low-resource languages, namely Wolof and Yoruba.
Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. Their success, however, heavily depends on the availability of many annotated image-caption datasets for pretraining, where the t
exts are not necessarily in English. Although we can utilize machine translation (MT) tools to translate non-English text to English, the performance still largely relies on MT's quality and may suffer from high latency problems in real-world applications. This paper proposes a new approach to learn cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. We seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual and parallel corpus. We show that our approach achieves SOTA performance in retrieval tasks on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions.
This paper presents the submission of team GUCLASP to SIGMORPHON 2021 Shared Task on Generalization in Morphological Inflection Generation. We develop a multilingual model for Morphological Inflection and primarily focus on improving the model by using various training strategies to improve accuracy and generalization across languages.
The choice of parameter sharing strategy in multilingual machine translation models determines how optimally parameter space is used and hence, directly influences ultimate translation quality. Inspired by linguistic trees that show the degree of rel
atedness between different languages, the new general approach to parameter sharing in multilingual machine translation was suggested recently. The main idea is to use these expert language hierarchies as a basis for multilingual architecture: the closer two languages are, the more parameters they share. In this work, we test this idea using the Transformer architecture and show that despite the success in previous work there are problems inherent to training such hierarchical models. We demonstrate that in case of carefully chosen training strategy the hierarchical architecture can outperform bilingual models and multilingual models with full parameter sharing.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
