Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userPaired Examples as Indirect Supervision in Latent Decision Models
أمثلة مقترنة على أنها إشراف غير مباشر في نماذج القرارات الكامنة
249
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
النماذج التركيبية المنظمة جذابة لأنها تتحلل صراحة المشاكل وتوفير مخرجات متوسطة تفسير تفسر الثقة في أن النموذج ليس مجرد إزالته على القطع الأثرية للبيانات. ومع ذلك، فإن تعلم هذه النماذج صعبة، ومع ذلك، نظرا لأن الإشراف على المهمة النهائية يوفر فقط إشارة غير مباشرة ضعيفة حول القيم التي يجب أن تتخذ القرارات الكامنة. غالبا ما يؤدي ذلك إلى فشل النموذج في تعلم كيفية تنفيذ المهام الوسيطة بشكل صحيح. في هذا العمل، نقدم طريقة للاستفادة من الأمثلة المقترنة التي توفر إشارات أقوى لتعلم القرارات الكامنة. عندما تتخلى أمثلة تدريبية ذات صلة بالحرارة الداخلية، نضيف هدف تدريب إضافي لتشجيع الاتساق بين قراراتهم الكامنة. لا يتطلب مثل هذا الهدف إشراف خارجي لقيم الإخراج الكامن، أو حتى المهمة النهائية، حتى الآن يوفر إشارة تدريب إضافية إلى ذلك من خلال أمثلة تدريب فردية أنفسهم. نحن نطبق طريقتنا لتحسين سؤال التركيبي الرد باستخدام شبكات الوحدات النمطية العصبية على Dropet DataSet. نستكشف ثلاث طرق للحصول على أسئلة مقترنة في قطرة: (أ) اكتشاف أمثلة مقترنة بشكل طبيعي داخل DataSet، (ب) بناء أمثلة مقترنة باستخدام القوالب، و (ج) إنشاء أمثلة مقنعة باستخدام نموذج جيل سؤال. إننا نوضح تجريبيا أن نهجنا المقترح يحسن التعميم داخل التوزيع ويؤدي إلى تصحيح تنبؤات القرارات الكامنة.
Compositional, structured models are appealing because they explicitly decompose problems and provide interpretable intermediate outputs that give confidence that the model is not simply latching onto data artifacts. Learning these models is challenging, however, because end-task supervision only provides a weak indirect signal on what values the latent decisions should take. This often results in the model failing to learn to perform the intermediate tasks correctly. In this work, we introduce a way to leverage paired examples that provide stronger cues for learning latent decisions. When two related training examples share internal substructure, we add an additional training objective to encourage consistency between their latent decisions. Such an objective does not require external supervision for the values of the latent output, or even the end task, yet provides an additional training signal to that provided by individual training examples themselves. We apply our method to improve compositional question answering using neural module networks on the DROP dataset. We explore three ways to acquire paired questions in DROP: (a) discovering naturally occurring paired examples within the dataset, (b) constructing paired examples using templates, and (c) generating paired examples using a question generation model. We empirically demonstrate that our proposed approach improves both in- and out-of-distribution generalization and leads to correct latent decision predictions.
References used
https://aclanthology.org/
rate research
Read More
For over thirty years, researchers have developed and analyzed methods for latent tree induction as an approach for unsupervised syntactic parsing. Nonetheless, modern systems still do not perform well enough compared to their supervised counterparts
to have any practical use as structural annotation of text. In this work, we present a technique that uses distant supervision in the form of span constraints (i.e. phrase bracketing) to improve performance in unsupervised constituency parsing. Using a relatively small number of span constraints we can substantially improve the output from DIORA, an already competitive unsupervised parsing system. Compared with full parse tree annotation, span constraints can be acquired with minimal effort, such as with a lexicon derived from Wikipedia, to find exact text matches. Our experiments show span constraints based on entities improves constituency parsing on English WSJ Penn Treebank by more than 5 F1. Furthermore, our method extends to any domain where span constraints are easily attainable, and as a case study we demonstrate its effectiveness by parsing biomedical text from the CRAFT dataset.
To audit the robustness of named entity recognition (NER) models, we propose RockNER, a simple yet effective method to create natural adversarial examples. Specifically, at the entity level, we replace target entities with other entities of the same
semantic class in Wikidata; at the context level, we use pre-trained language models (e.g., BERT) to generate word substitutions. Together, the two levels of at- tack produce natural adversarial examples that result in a shifted distribution from the training data on which our target models have been trained. We apply the proposed method to the OntoNotes dataset and create a new benchmark named OntoRock for evaluating the robustness of existing NER models via a systematic evaluation protocol. Our experiments and analysis reveal that even the best model has a significant performance drop, and these models seem to memorize in-domain entity patterns instead of reasoning from the context. Our work also studies the effects of a few simple data augmentation methods to improve the robustness of NER models.
Variational autoencoders (VAEs) are widely used for latent variable modeling of text. We focus on variations that learn expressive prior distributions over the latent variable. We find that existing training strategies are not effective for learning
rich priors, so we propose adding the importance-sampled log marginal likelihood as a second term to the standard VAE objective to help when learning the prior. Doing so improves results for all priors evaluated, including a novel choice for sentence VAEs based on normalizing flows (NF). Priors parameterized with NF are no longer constrained to a specific distribution family, allowing a more flexible way to encode the data distribution. Our model, which we call FlowPrior, shows a substantial improvement in language modeling tasks compared to strong baselines. We demonstrate that FlowPrior learns an expressive prior with analysis and several forms of evaluation involving generation.
Common acquisition functions for active learning use either uncertainty or diversity sampling, aiming to select difficult and diverse data points from the pool of unlabeled data, respectively. In this work, leveraging the best of both worlds, we prop
ose an acquisition function that opts for selecting contrastive examples, i.e. data points that are similar in the model feature space and yet the model outputs maximally different predictive likelihoods. We compare our approach, CAL (Contrastive Active Learning), with a diverse set of acquisition functions in four natural language understanding tasks and seven datasets. Our experiments show that CAL performs consistently better or equal than the best performing baseline across all tasks, on both in-domain and out-of-domain data. We also conduct an extensive ablation study of our method and we further analyze all actively acquired datasets showing that CAL achieves a better trade-off between uncertainty and diversity compared to other strategies.
Story generation is an open-ended and subjective task, which poses a challenge for evaluating story generation models. We present Choose Your Own Adventure, a collaborative writing setup for pairwise model evaluation. Two models generate suggestions
to people as they write a short story; we ask writers to choose one of the two suggestions, and we observe which model's suggestions they prefer. The setup also allows further analysis based on the revisions people make to the suggestions. We show that these measures, combined with automatic metrics, provide an informative picture of the models' performance, both in cases where the differences in generation methods are small (nucleus vs. top-k sampling) and large (GPT2 vs. Fusion models).
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
