Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUnsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data
تكبير البيانات غير المزعجة مع زيادة ساذجة وبدون بيانات غير مسافة
319
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعظيم البيانات غير المزعجة (UDA) هي تقنية شبه بيئية تنطبق على فقدان الاتساق لمعاقبة الاختلافات بين تنبؤات النماذج على (أ) أمثلة ملحوظة (غير مسفحة)؛ و (ب) الأمثلة الواضحة المقابلة التي تم إنتاجها عبر تكبير البيانات. في حين أن UDA اكتسبت شعبية لتصنيف النصوص، فإن الأسئلة المفتوحة باقية من قرارات التصميم ضرورية وكيفية تمديد الطريقة لتسلسل مهام وضع العلامات. في هذه الورقة، نعيد فحص UDA وإظهار فعاليتها في العديد من المهام المتسلسلة. مساهمتنا الرئيسية هي دراسة تجريبية ل UDA لتأسيس مكونات الخوارزمية التي تمنح استحقاقات NLP. وخاصة، على الرغم من أن العمل السابق قد أكد على استخدام تقنيات تكبير ذكية بما في ذلك الترجمة ذات الترجمة المرجانية، نجد أن التناسق بين التنبؤات المخصصة للكلمات الملحوظة والمستبدلة غالبا ما تسفر عن فوائد قابلة للمقارنة (أو أكبر) مقارنة بنماذج الاضطرابات الأكثر تعقيدا. علاوة على ذلك، نجد أن تطبيق فقدان اتساق UDA يوفر مكاسب ذات مغزى دون أي بيانات غير قابلة للتحقيق على الإطلاق، أي في إعداد قياسي إشرافي. باختصار، لا تحتاج UDA إلى عدم إدراكها لتحقيق الكثير من فوائدها المذكورة، ولا تتطلب تكبير بيانات معقدة لتكون فعالة.
Unsupervised Data Augmentation (UDA) is a semisupervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding noised' examples produced via data augmentation. While UDA has gained popularity for text classification, open questions linger over which design decisions are necessary and how to extend the method to sequence labeling tasks. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. Our main contribution is an empirical study of UDA to establish which components of the algorithm confer benefits in NLP. Notably, although prior work has emphasized the use of clever augmentation techniques including back-translation, we find that enforcing consistency between predictions assigned to observed and randomly substituted words often yields comparable (or greater) benefits compared to these more complex perturbation models. Furthermore, we find that applying UDA's consistency loss affords meaningful gains without any unlabeled data at all, i.e., in a standard supervised setting. In short, UDA need not be unsupervised to realize much of its noted benefits, and does not require complex data augmentation to be effective.
References used
https://aclanthology.org/
rate research
Read More
We address the compositionality challenge presented by the SCAN benchmark. Using data augmentation and a modification of the standard seq2seq architecture with attention, we achieve SOTA results on all the relevant tasks from the benchmark, showing t
he models can generalize to words used in unseen contexts. We propose an extension of the benchmark by a harder task, which cannot be solved by the proposed method.
Previous works on syntactically controlled paraphrase generation heavily rely on large-scale parallel paraphrase data that is not easily available for many languages and domains. In this paper, we take this research direction to the extreme and inves
tigate whether it is possible to learn syntactically controlled paraphrase generation with nonparallel data. We propose a syntactically-informed unsupervised paraphrasing model based on conditional variational auto-encoder (VAE) which can generate texts in a specified syntactic structure. Particularly, we design a two-stage learning method to effectively train the model using non-parallel data. The conditional VAE is trained to reconstruct the input sentence according to the given input and its syntactic structure. Furthermore, to improve the syntactic controllability and semantic consistency of the pre-trained conditional VAE, we fine-tune it using syntax controlling and cycle reconstruction learning objectives, and employ Gumbel-Softmax to combine these new learning objectives. Experiment results demonstrate that the proposed model trained only on non-parallel data is capable of generating diverse paraphrases with specified syntactic structure. Additionally, we validate the effectiveness of our method for generating syntactically adversarial examples on the sentiment analysis task.
We present an algorithm based on multi-layer transformers for identifying Adverse Drug Reactions (ADR) in social media data. Our model relies on the properties of the problem and the characteristics of contextual word embeddings to extract two views
from documents. Then a classifier is trained on each view to label a set of unlabeled documents to be used as an initializer for a new classifier in the other view. Finally, the initialized classifier in each view is further trained using the initial training examples. We evaluated our model in the largest publicly available ADR dataset. The experiments testify that our model significantly outperforms the transformer-based models pretrained on domain-specific data.
We propose a data augmentation method for neural machine translation. It works by interpreting language models and phrasal alignment causally. Specifically, it creates augmented parallel translation corpora by generating (path-specific) counterfactua
l aligned phrases. We generate these by sampling new source phrases from a masked language model, then sampling an aligned counterfactual target phrase by noting that a translation language model can be interpreted as a Gumbel-Max Structural Causal Model (Oberst and Sontag, 2019). Compared to previous work, our method takes both context and alignment into account to maintain the symmetry between source and target sequences. Experiments on IWSLT'15 English → Vietnamese, WMT'17 English → German, WMT'18 English → Turkish, and WMT'19 robust English → French show that the method can improve the performance of translation, backtranslation and translation robustness.
Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed. We focus here on gloss-to-text
translation, which we treat as a low-resource neural machine translation (NMT) problem. However, unlike traditional low resource NMT, gloss-to-text translation differs because gloss-text pairs often have a higher lexical overlap and lower syntactic overlap than pairs of spoken languages. We exploit this lexical overlap and handle syntactic divergence by proposing two rule-based heuristics that generate pseudo-parallel gloss-text pairs from monolingual spoken language text. By pre-training on this synthetic data, we improve translation from American Sign Language (ASL) to English and German Sign Language (DGS) to German by up to 3.14 and 2.20 BLEU, respectively.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
