Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new usermixSeq: A Simple Data Augmentation Methodfor Neural Machine Translation
تسك: طريقة تكبير البيانات بسيطة للترجمة الآلية العصبية
427
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعزز البيانات، التي تشير إلى معالجة المدخلات (على سبيل المثال، إضافة ضوضاء عشوائية، اخفاء أجزاء محددة) لتكبير مجموعة البيانات، تم اعتمادها على نطاق واسع في تعلم الجهاز.تعمل معظم تقنيات تكبير البيانات على إدخال واحد، مما يحد من تنوع كوربوس التدريب.في هذه الورقة، نقترح تقنية بسيطة ولكنها فعالة لتكبير البيانات للترجمة الآلية العصبية، mixseq، والتي تعمل على مدخلات متعددة وأهدافها المقابلة لها.على وجه التحديد، نقوم بشكل عشوائي بتحديد تسلسل مدخلتين بشكل عشوائي، وتسلسلها معا كإدخال أطول كما أن تسلسل المستحضرات المستهدفة المقابلة للهدف الموسع، ونماذج القطار على مجموعة البيانات المستعملة.تثبت التجارب في تسع مهام ترجمة آلية أن هذه الطريقة Asimple تعزز الخط الأساس عن طريق هامش غير تافهة.يمكن دمج طريقةنا مع طرق تكبير البيانات المستندة إلى الإدخال الفردي للحصول على مزيد من التحسينات.
Data augmentation, which refers to manipulating the inputs (e.g., adding random noise,masking specific parts) to enlarge the dataset,has been widely adopted in machine learning. Most data augmentation techniques operate on a single input, which limits the diversity of the training corpus. In this paper, we propose a simple yet effective data augmentation technique for neural machine translation, mixSeq, which operates on multiple inputs and their corresponding targets. Specifically, we randomly select two input sequences,concatenate them together as a longer input aswell as their corresponding target sequencesas an enlarged target, and train models on theaugmented dataset. Experiments on nine machine translation tasks demonstrate that such asimple method boosts the baselines by a non-trivial margin. Our method can be further combined with single input based data augmentation methods to obtain further improvements.
References used
https://aclanthology.org/
rate research
Read More
We propose a data augmentation method for neural machine translation. It works by interpreting language models and phrasal alignment causally. Specifically, it creates augmented parallel translation corpora by generating (path-specific) counterfactua
l aligned phrases. We generate these by sampling new source phrases from a masked language model, then sampling an aligned counterfactual target phrase by noting that a translation language model can be interpreted as a Gumbel-Max Structural Causal Model (Oberst and Sontag, 2019). Compared to previous work, our method takes both context and alignment into account to maintain the symmetry between source and target sequences. Experiments on IWSLT'15 English → Vietnamese, WMT'17 English → German, WMT'18 English → Turkish, and WMT'19 robust English → French show that the method can improve the performance of translation, backtranslation and translation robustness.
Recently, neural machine translation is widely used for its high translation accuracy, but it is also known to show poor performance at long sentence translation. Besides, this tendency appears prominently for low resource languages. We assume that t
hese problems are caused by long sentences being few in the train data. Therefore, we propose a data augmentation method for handling long sentences. Our method is simple; we only use given parallel corpora as train data and generate long sentences by concatenating two sentences. Based on our experiments, we confirm improvements in long sentence translation by proposed data augmentation despite the simplicity. Moreover, the proposed method improves translation quality more when combined with back-translation.
We observe that the development cross-entropy loss of supervised neural machine translation models scales like a power law with the amount of training data and the number of non-embedding parameters in the model. We discuss some practical implication
s of these results, such as predicting BLEU achieved by large scale models and predicting the ROI of labeling data in low-resource language pairs.
This paper describes the participation of the UoB-NLP team in the ProfNER-ST shared subtask 7a. The task was aimed at detecting the mention of professions in social media text. Our team experimented with two methods of improving the performance of pr
e-trained models: Specifically, we experimented with data augmentation through translation and the merging of multiple language inputs to meet the objective of the task. While the best performing model on the test data consisted of mBERT fine-tuned on augmented data using back-translation, the improvement is minor possibly because multi-lingual pre-trained models such as mBERT already have access to the kind of information provided through back-translation and bilingual data.
For Japanese-to-English translation, zero pronouns in Japanese pose a challenge, since the model needs to infer and produce the corresponding pronoun in the target side of the English sentence. However, although fully resolving zero pronouns often ne
eds discourse context, in some cases, the local context within a sentence gives clues to the inference of the zero pronoun. In this study, we propose a data augmentation method that provides additional training signals for the translation model to learn correlations between local context and zero pronouns. We show that the proposed method significantly improves the accuracy of zero pronoun translation with machine translation experiments in the conversational domain.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
