Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Comparison of Different NMT Approaches to Low-Resource Dutch-Albanian Machine Translation
مقارنة بين نهج NMT المختلفة إلى الترجمة ذات الموارد الهولندية-الألبانية المنخفضة
262
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يمكن فهم لغات الموارد المنخفضة كنغات أكثر شحيحة، وأقل دراستها، أقل امتيازا، أقل شيوعا، والتي تكون أقل شيوعا والتي توجد فيها موارد أقل (Singh، 2008؛ Cieri et al.، 2016؛ Magueresse et al.، 2020) وبعد يركز البحث والتكنولوجيا لمعالجة اللغة الطبيعية (NLP) بشكل أساسي على تلك اللغات التي توجد بها مجموعات بيانات كبيرة متاحة. لتوضيح الاختلافات في توافر البيانات: هناك 6 ملايين مقالة في ويكيبيديا المتاحة للغة الإنجليزية، 2 مليون للهولندية، ومجرد 82 ألف للألبانية. تصبح قضية البيانات الشحيحة واضحة بشكل متزايد عندما تكون مجموعات البيانات المتوازية الكبيرة مطلوبة للتطبيقات مثل الترجمة الآلية العصبية (NMT). في هذا العمل، يمكننا التحقيق في أي مدى من الممكن الترجمة بين الألبانية (SQ) والهولندية (NL) مقارنة نموذج واحد إلى واحد (SQ↔AL)، نهج يستند إلى موارد منخفضة الموارد (الإنجليزية (EN) Pivot) والترجمة الصفرية بالرصاص (ZST) (جونسون وآخرون، 2016؛ نظام ماتوني وآخرون.، 2017). من تجاربنا، فإنه ينتج عن تفوق نموذج EN-PIVOT على حد سواء من طراز Zst المباشر. منذ غالبا ما تكون كميات صغيرة من البيانات الموازية متاحة لغات الموارد المنخفضة أو الإعدادات المنخفضة، أجريت التجارب باستخدام مجموعات صغيرة من بيانات NL↔SQ الموازية. بدا أن Zst هو أسوأ نماذج أداء. حتى عندما تمت إضافة البيانات الموازية المتاحة (nl↔sq)، أي في إعداد قليل من اللقطة (FST)، ظلت أسوأ نظام أداء وفقا ل Automatic (Bleu and Ter) والتقييم البشري.
Low-resource languages can be understood as languages that are more scarce, less studied, less privileged, less commonly taught and for which there are less resources available (Singh, 2008; Cieri et al., 2016; Magueresse et al., 2020). Natural Language Processing (NLP) research and technology mainly focuses on those languages for which there are large data sets available. To illustrate differences in data availability: there are 6 million Wikipedia articles available for English, 2 million for Dutch, and merely 82 thousand for Albanian. The scarce data issue becomes increasingly apparent when large parallel data sets are required for applications such as Neural Machine Translation (NMT). In this work, we investigate to what extent translation between Albanian (SQ) and Dutch (NL) is possible comparing a one-to-one (SQ↔AL) model, a low-resource pivot-based approach (English (EN) as pivot) and a zero-shot translation (ZST) (Johnson et al., 2016; Mattoni et al., 2017) system. From our experiments, it results that the EN-pivot-model outperforms both the direct one-to-one and the ZST model. Since often, small amounts of parallel data are available for low-resource languages or settings, experiments were conducted using small sets of parallel NL↔SQ data. The ZST appeared to be the worst performing models. Even when the available parallel data (NL↔SQ) was added, i.e. in a few-shot setting (FST), it remained the worst performing system according to the automatic (BLEU and TER) and human evaluation.
References used
https://aclanthology.org/
rate research
Read More
A bigger is better'' explosion in the number of parameters in deep neural networks has made it increasingly challenging to make state-of-the-art networks accessible in compute-restricted environments. Compression techniques have taken on renewed impo
rtance as a way to bridge the gap. However, evaluation of the trade-offs incurred by popular compression techniques has been centered on high-resource datasets. In this work, we instead consider the impact of compression in a data-limited regime. We introduce the term low-resource double bind to refer to the co-occurrence of data limitations and compute resource constraints. This is a common setting for NLP for low-resource languages, yet the trade-offs in performance are poorly studied. Our work offers surprising insights into the relationship between capacity and generalization in data-limited regimes for the task of machine translation. Our experiments on magnitude pruning for translations from English into Yoruba, Hausa, Igbo and German show that in low-resource regimes, sparsity preserves performance on frequent sentences but has a disparate impact on infrequent ones. However, it improves robustness to out-of-distribution shifts, especially for datasets that are very distinct from the training distribution. Our findings suggest that sparsity can play a beneficial role at curbing memorization of low frequency attributes, and therefore offers a promising solution to the low-resource double bind.
In this work, we investigate methods for the challenging task of translating between low- resource language pairs that exhibit some level of similarity. In particular, we consider the utility of transfer learning for translating between several Indo-
European low-resource languages from the Germanic and Romance language families. In particular, we build two main classes of transfer-based systems to study how relatedness can benefit the translation performance. The primary system fine-tunes a model pre-trained on a related language pair and the contrastive system fine-tunes one pre-trained on an unrelated language pair. Our experiments show that although relatedness is not necessary for transfer learning to work, it does benefit model performance.
In this paper and we explore different techniques of overcoming the challenges of low-resource in Neural Machine Translation (NMT) and specifically focusing on the case of English-Marathi NMT. NMT systems require a large amount of parallel corpora to
obtain good quality translations. We try to mitigate the low-resource problem by augmenting parallel corpora or by using transfer learning. Techniques such as Phrase Table Injection (PTI) and back-translation and mixing of language corpora are used for enhancing the parallel data; whereas pivoting and multilingual embeddings are used to leverage transfer learning. For pivoting and Hindi comes in as assisting language for English-Marathi translation. Compared to baseline transformer model and a significant improvement trend in BLEU score is observed across various techniques. We have done extensive manual and automatic and qualitative evaluation of our systems. Since the trend in Machine Translation (MT) today is post-editing and measuring of Human Effort Reduction (HER) and we have given our preliminary observations on Translation Edit Rate (TER) vs. BLEU score study and where TER is regarded as a measure of HER.
For most language combinations and parallel data is either scarce or simply unavailable. To address this and unsupervised machine translation (UMT) exploits large amounts of monolingual data by using synthetic data generation techniques such as back-
translation and noising and while self-supervised NMT (SSNMT) identifies parallel sentences in smaller comparable data and trains on them. To this date and the inclusion of UMT data generation techniques in SSNMT has not been investigated. We show that including UMT techniques into SSNMT significantly outperforms SSNMT (up to +4.3 BLEU and af2en) as well as statistical (+50.8 BLEU) and hybrid UMT (+51.5 BLEU) baselines on related and distantly-related and unrelated language pairs.
Sentence weighting is a simple and powerful domain adaptation technique. We carry out domain classification for computing sentence weights with 1) language model cross entropy difference 2) a convolutional neural network 3) a Recursive Neural Tensor
Network. We compare these approaches with regard to domain classification accuracy and and study the posterior probability distributions. Then we carry out NMT experiments in the scenario where we have no in-domain parallel corpora and and only very limited in-domain monolingual corpora. Here and we use the domain classifier to reweight the sentences of our out-of-domain training corpus. This leads to improvements of up to 2.1 BLEU for German to English translation.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
