Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization
Crossvqa: توليد المعايير بشكل متقن لاختبار نظام VQA بشكل منهجي
321
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تحدي واحد في تقييم نماذج الإجابة عن السؤال المرئي (VQA) في إعداد التكيف عبر DataSet هو أن التحولات التوزيع متعددة الوسائط متعددة، مما يجعل من الصعب تحديد ما إذا كانت التحولات في ميزات مرئية أو لغة تلعب دورا رئيسيا. في هذه الورقة، نقترح إطارا شبه أوتوماتيكي لإنشاء تحولات محددة من خلال إدخال وحدة نمطية لجيل الإجابات مرئية يمكن السيطرة عليها (VQAG) قادرة على توليد أزواج للإجابة على الأسئلة ذات الصلة والتنوع مع نمط البيانات المطلوب. نستخدمها لإنشاء crossvqa، وهي مجموعة من تقسيم الاختبار لتقييم مجموعات بيانات VQA2، VizWiz، وفتح الصور المفتوحة. نحن نقدم تحليلا لمجموعات البيانات التي تم إنشاؤها وإظهار فائدتها باستخدامها لتقييم العديد من أنظمة VQA الحديثة. اكتشاف واحد مهم هو أن التحولات المرئية في VQA عبر DataSet يهم أكثر من التحولات اللغوية. على نطاق أوسع، نقدم إطارا قابل للتطوير لتقييم الجهاز بشكل منهجي مع التدخل البشري القليل.
One challenge in evaluating visual question answering (VQA) models in the cross-dataset adaptation setting is that the distribution shifts are multi-modal, making it difficult to identify if it is the shifts in visual or language features that play a key role. In this paper, we propose a semi-automatic framework for generating disentangled shifts by introducing a controllable visual question-answer generation (VQAG) module that is capable of generating highly-relevant and diverse question-answer pairs with the desired dataset style. We use it to create CrossVQA, a collection of test splits for assessing VQA generalization based on the VQA2, VizWiz, and Open Images datasets. We provide an analysis of our generated datasets and demonstrate its utility by using them to evaluate several state-of-the-art VQA systems. One important finding is that the visual shifts in cross-dataset VQA matter more than the language shifts. More broadly, we present a scalable framework for systematically evaluating the machine with little human intervention.
References used
https://aclanthology.org/
rate research
Read More
We propose EASE, a simple diagnostic tool for Visual Question Answering (VQA) which quantifies the difficulty of an image, question sample. EASE is based on the pattern of answers provided by multiple annotators to a given question. In particular, it
considers two aspects of the answers: (i) their Entropy; (ii) their Semantic content. First, we prove the validity of our diagnostic to identify samples that are easy/hard for state-of-art VQA models. Second, we show that EASE can be successfully used to select the most-informative samples for training/fine-tuning. Crucially, only information that is readily available in any VQA dataset is used to compute its scores.
MiniVQA is a Jupyter notebook to build a tailored VQA competition for your students. The resource creates all the needed resources to create a classroom competition that engages and inspires your students on the free, self-service Kaggle platform. InClass competitions make machine learning fun!.
Although pre-training models have achieved great success in dialogue generation, their performance drops dramatically when the input contains an entity that does not appear in pre-training and fine-tuning datasets (unseen entity). To address this iss
ue, existing methods leverage an external knowledge base to generate appropriate responses. In real-world practical, the entity may not be included by the knowledge base or suffer from the precision of knowledge retrieval. To deal with this problem, instead of introducing knowledge base as the input, we force the model to learn a better semantic representation by predicting the information in the knowledge base, only based on the input context. Specifically, with the help of a knowledge base, we introduce two auxiliary training objectives: 1) Interpret Masked Word, which conjectures the meaning of the masked entity given the context; 2) Hypernym Generation, which predicts the hypernym of the entity based on the context. Experiment results on two dialogue corpus verify the effectiveness of our methods under both knowledge available and unavailable settings.
يعرف التوحد بأنه أحد الاضطرابات النمائية الشاملة يتسم بقصور نوعي في مهارات التواصل والانتباه ومهارات التفاعل الاجتماعي
Abductive reasoning starts from some observations and aims at finding the most plausible explanation for these observations. To perform abduction, humans often make use of temporal and causal inferences, and knowledge about how some hypothetical situ
ation can result in different outcomes. This work offers the first study of how such knowledge impacts the Abductive NLI task -- which consists in choosing the more likely explanation for given observations. We train a specialized language model LMI that is tasked to generate what could happen next from a hypothetical scenario that evolves from a given event. We then propose a multi-task model MTL to solve the Abductive NLI task, which predicts a plausible explanation by a) considering different possible events emerging from candidate hypotheses -- events generated by LMI -- and b) selecting the one that is most similar to the observed outcome. We show that our MTL model improves over prior vanilla pre-trained LMs fine-tuned on Abductive NLI. Our manual evaluation and analysis suggest that learning about possible next events from different hypothetical scenarios supports abductive inference.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
