Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCompositional Data and Task Augmentation for Instruction Following
البيانات التركيبية وتعزيز المهمة للحصول على التعليمات التالية
304
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يتطلب تنفيذ تعليمات اللغة الطبيعية في مجال أساسي جسديا نموذجا يفهم كل من المفاهيم المكانية مثل اليسار من "" وما فوق ""، واللغة التركيبية المستخدمة لتحديد المعالم وتعيين التعليمات المتعلقة بها. في هذه الورقة، ندرس فهم التعليمات في المجال العالمي كتل. بالنظر إلى ترتيب أولي من الكتل وتعليم اللغة الطبيعية، يقوم النظام بتنفيذ التعليمات عن طريق التلاعب بالكتل المحددة. تتألف التعليمات التركيبية للغاية من مكونات ذرية وفهم هذه المكونات خطوة ضرورية لتنفيذ التعليمات. نظرا لأنه أثناء وجود تدريب نهاية إلى نهائي (يشرف عليه موقع الكتلة الصحيحة فقط) فشل في معالجة تحديات هذه المهمة ويعمل بشكل سيء على التعليمات التي تنطوي على مكون ذرية واحدة، يمكن استخدام الإشارات المساعدة الخالية من المعرفة لتحسين الأداء بشكل كبير من خلال توفير الإشراف على مكونات التعليمات. على وجه التحديد، نوفر إشارات تهدف إلى مساعدة النموذج تدريجيا على فهم مكونات التعليمات التركيبية، وكذلك تلك التي تساعدها على فهم المفاهيم المكانية بشكل أفضل، وإظهار فائدةها للمهمة الشاملة لمجموعات البيانات واثنين من نماذج الفن (SOTA)، خاصة عندما تكون بيانات التدريب محدودة --- وهي المعتادة في هذه المهام.
Executing natural language instructions in a physically grounded domain requires a model that understands both spatial concepts such as left of'' and above'', and the compositional language used to identify landmarks and articulate instructions relative to them. In this paper, we study instruction understanding in the blocks world domain. Given an initial arrangement of blocks and a natural language instruction, the system executes the instruction by manipulating selected blocks. The highly compositional instructions are composed of atomic components and understanding these components is a necessary step to executing the instruction. We show that while end-to-end training (supervised only by the correct block location) fails to address the challenges of this task and performs poorly on instructions involving a single atomic component, knowledge-free auxiliary signals can be used to significantly improve performance by providing supervision for the instruction's components. Specifically, we generate signals that aim at helping the model gradually understand components of the compositional instructions, as well as those that help it better understand spatial concepts, and show their benefit to the overall task for two datasets and two state-of-the-art (SOTA) models, especially when the training data is limited---which is usual in such tasks.
References used
https://aclanthology.org/
rate research
Read More
Standard architectures used in instruction following often struggle on novel compositions of subgoals (e.g. navigating to landmarks or picking up objects) observed during training. We propose a modular architecture for following natural language inst
ructions that describe sequences of diverse subgoals. In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type. A sequence of modules to execute is chosen by learning to segment the instructions and predicting a subgoal type for each segment. When compared to standard, non-modular sequence-to-sequence approaches on ALFRED, a challenging instruction following benchmark, we find that modularization improves generalization to novel subgoal compositions, as well as to environments unseen in training.
Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence. In this paper, we focus on instruction understanding in the blocks world domain and investigate the language understand
ing abilities of two top-performing systems for the task. We aim to understand if the test performance of these models indicates an understanding of the spatial domain and of the natural language instructions relative to it, or whether they merely over-fit spurious signals in the dataset. We formulate a set of expectations one might have from an instruction following model and concretely characterize the different dimensions of robustness such a model should possess. Despite decent test performance, we find that state-of-the-art models fall short of these expectations and are extremely brittle. We then propose a learning strategy that involves data augmentation and show through extensive experiments that the proposed learning strategy yields models that are competitive on the original test set while satisfying our expectations much better.
Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed. We focus here on gloss-to-text
translation, which we treat as a low-resource neural machine translation (NMT) problem. However, unlike traditional low resource NMT, gloss-to-text translation differs because gloss-text pairs often have a higher lexical overlap and lower syntactic overlap than pairs of spoken languages. We exploit this lexical overlap and handle syntactic divergence by proposing two rule-based heuristics that generate pseudo-parallel gloss-text pairs from monolingual spoken language text. By pre-training on this synthetic data, we improve translation from American Sign Language (ASL) to English and German Sign Language (DGS) to German by up to 3.14 and 2.20 BLEU, respectively.
Coupled with the availability of large scale datasets, deep learning architectures have enabled rapid progress on the Question Answering task. However, most of those datasets are in English, and the performances of state-of-the-art multilingual model
s are significantly lower when evaluated on non-English data. Due to high data collection costs, it is not realistic to obtain annotated data for each language one desires to support. We propose a method to improve the Cross-lingual Question Answering performance without requiring additional annotated data, leveraging Question Generation models to produce synthetic samples in a cross-lingual fashion. We show that the proposed method allows to significantly outperform the baselines trained on English data only. We report a new state-of-the-art on four datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr).
In this paper we explore a very simple neural approach to mapping orthography to phonetic transcription in a low-resource context. The basic idea is to start from a baseline system and focus all efforts on data augmentation. We will see that some techniques work, but others do not.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
