Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEnriching the Transformer with Linguistic Factors for Low-Resource Machine Translation
إثراء المحول مع العوامل اللغوية للترجمة ذات الجهاز المنخفض
112
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
من المعروف أن ميزات كلمة مثل المعلومات اللغوية التي تشير إلى رموز المصدر التي تشير إلى رموز المصدر، لتحسين نتائج أنظمة الترجمة الآلية العصبية في بعض الإعدادات، وعادة ما تكون في البنى المتكررة. تقترح هذه الدراسة تعزيز هندسة الترجمة الآلية الحالية للدولة القصيرة، والمحول، بحيث يسمح بإدخال المعرفة الخارجية. على وجه الخصوص، يستخدم التعديل المقترح لدينا، المحولات العامل، العوامل اللغوية التي تدرس معرفة إضافية في نظام الترجمة الآلي. بصرف النظر عن استخدام أنواع مختلفة من الميزات، ندرس تأثير التكوينات المعمارية المختلفة. على وجه التحديد، نقوم بتحليل أداء الجمع بين الكلمات والميزات على مستوى التضمين أو على مستوى التشفير، ونحن نقوم بتجربة استراتيجيتين مزيج مختلفين. مع تكوين أفضل تم العثور عليه، نعرض تحسينات من 0.8 بلو عبر محول الأساس في مهمة IWSLT الألمانية إلى الإنجليزية. علاوة على ذلك، نقوم بتجربة معيار فلوريس الإنجليزي إلى النيبالي الأكثر تحديا، والذي يشمل كل من اللغات المنخفضة الموارد والبعيدة للغاية، والحصول على تحسين 1.2 بلو
Introducing factors, that is to say, word features such as linguistic information referring to the source tokens, is known to improve the results of neural machine translation systems in certain settings, typically in recurrent architectures. This study proposes enhancing the current state-of-the-art neural machine translation architecture, the Transformer, so that it allows to introduce external knowledge. In particular, our proposed modification, the Factored Transformer, uses linguistic factors that insert additional knowledge into the machine translation system. Apart from using different kinds of features, we study the effect of different architectural configurations. Specifically, we analyze the performance of combining words and features at the embedding level or at the encoder level, and we experiment with two different combination strategies. With the best-found configuration, we show improvements of 0.8 BLEU over the baseline Transformer in the IWSLT German-to-English task. Moreover, we experiment with the more challenging FLoRes English-to-Nepali benchmark, which includes both extremely low-resourced and very distant languages, and obtain an improvement of 1.2 BLEU
References used
https://aclanthology.org/
rate research
Read More
It has been widely recognized that syntax information can help end-to-end neural machine translation (NMT) systems to achieve better translation. In order to integrate dependency information into Transformer based NMT, existing approaches either expl
oit words' local head-dependent relations, ignoring their non-local neighbors carrying important context; or approximate two words' syntactic relation by their relative distance on the dependency tree, sacrificing exactness. To address these issues, we propose global positional encoding for dependency tree, a new scheme that facilitates syntactic relation modeling between any two words with keeping exactness and without immediate neighbor constraint. Experiment results on NC11 German→English, English→German and WMT English→German datasets show that our approach is more effective than the above two strategies. In addition, our experiments quantitatively show that compared with higher layers, lower layers of the model are more proper places to incorporate syntax information in terms of each layer's preference to the syntactic pattern and the final performance.
In the paper, we present the process of adding morphological information to the Polish WordNet (plWordNet). We describe the reasons for this connection and the intuitions behind it. We also draw attention to the specificity of the Polish morphology.
We show in which tasks the morphological information is important and how the methods can be developed by extending them to include combined morphological information based on WordNet.
Neural Machine Translation (NMT) for Low Resource Languages (LRL) is often limited by the lack of available training data, making it necessary to explore additional techniques to improve translation quality. We propose the use of the Prefix-Root-Post
fix-Encoding (PRPE) subword segmentation algorithm to improve translation quality for LRLs, using two agglutinative languages as case studies: Quechua and Indonesian. During the course of our experiments, we reintroduce a parallel corpus for Quechua-Spanish translation that was previously unavailable for NMT. Our experiments show the importance of appropriate subword segmentation, which can go as far as improving translation quality over systems trained on much larger quantities of data. We show this by achieving state-of-the-art results for both languages, obtaining higher BLEU scores than large pre-trained models with much smaller amounts of data.
Taxonomies are symbolic representations of hierarchical relationships between terms or entities. While taxonomies are useful in broad applications, manually updating or maintaining them is labor-intensive and difficult to scale in practice. Conventio
nal supervised methods for this enrichment task fail to find optimal parents of new terms in low-resource settings where only small taxonomies are available because of overfitting to hierarchical relationships in the taxonomies. To tackle the problem of low-resource taxonomy enrichment, we propose Musubu, an efficient framework for taxonomy enrichment in low-resource settings with pretrained language models (LMs) as knowledge bases to compensate for the shortage of information. Musubu leverages an LM-based classifier to determine whether or not inputted term pairs have hierarchical relationships. Musubu also utilizes Hearst patterns to generate queries to leverage implicit knowledge from the LM efficiently for more accurate prediction. We empirically demonstrate the effectiveness of our method in extensive experiments on taxonomies from both a SemEval task and real-world retailer datasets.
The choice of parameter sharing strategy in multilingual machine translation models determines how optimally parameter space is used and hence, directly influences ultimate translation quality. Inspired by linguistic trees that show the degree of rel
atedness between different languages, the new general approach to parameter sharing in multilingual machine translation was suggested recently. The main idea is to use these expert language hierarchies as a basis for multilingual architecture: the closer two languages are, the more parameters they share. In this work, we test this idea using the Transformer architecture and show that despite the success in previous work there are problems inherent to training such hierarchical models. We demonstrate that in case of carefully chosen training strategy the hierarchical architecture can outperform bilingual models and multilingual models with full parameter sharing.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
