Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userMM-AVS: A Full-Scale Dataset for Multi-modal Summarization
MM-AVS: مجموعة بيانات كاملة الحجم لتلخيص متعدد الوسائط
295
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تصبح التلخصات متعددة الوسائط ذات أهمية متزايدة لأنها هي أساس الإجابة على السؤال والبحث على شبكة الإنترنت والعديد من المهام الأخرى المصب الأخرى.ومع ذلك، فإن موادها التعليمية تفتقر إلى منظمة شاملة من خلال دمج الموارد من الطرائق المختلفة، مما يتخلف عن التقدم المحرز البحثي في هذا المجال.في هذه الدراسة، نطلق عن مجموعة بيانات متعددة الوسائط واسعة النطاق تجمع الوثائق والملخصات والصور والتسميات التوضيحية ومقاطع الفيديو والصوتيات والمنصات واللقب باللغة الإنجليزية من CNN و Daily Mail.لدينا أفضل المعرفة لدينا، هذه هي المجموعة الأولى التي تدعم جميع الطرائق وتشمل ما يقرب من جميع أنواع المواد المتاحة في هذا المجتمع.بالإضافة إلى ذلك، نحن ابتكر نموذج خط الأساس استنادا إلى مجموعة البيانات الجديدة، والذي يستخدم آلية تقفيز تقترح حديثا على أساس النصوص.تتحقق النتائج التجريبية دور المساعدة الهامة للمعلومات الخارجية لتلخيص متعدد الوسائط.
Multimodal summarization becomes increasingly significant as it is the basis for question answering, Web search, and many other downstream tasks. However, its learning materials have been lacking a holistic organization by integrating resources from various modalities, thereby lagging behind the research progress of this field. In this study, we release a full-scale multimodal dataset comprehensively gathering documents, summaries, images, captions, videos, audios, transcripts, and titles in English from CNN and Daily Mail. To our best knowledge, this is the first collection that spans all modalities and nearly comprises all types of materials available in this community. In addition, we devise a baseline model based on the novel dataset, which employs a newly proposed Jump-Attention mechanism based on transcripts. The experimental results validate the important assistance role of the external information for multimodal summarization.
References used
https://aclanthology.org/
rate research
Read More
Abstract Aspect-based summarization is the task of generating focused summaries based on specific points of interest. Such summaries aid efficient analysis of text, such as quickly understanding reviews or opinions from different angles. However, due
to large differences in the type of aspects for different domains (e.g., sentiment, product features), the development of previous models has tended to be domain-specific. In this paper, we propose WikiAsp,1 a large-scale dataset for multi-domain aspect- based summarization that attempts to spur research in the direction of open-domain aspect-based summarization. Specifically, we build the dataset using Wikipedia articles from 20 different domains, using the section titles and boundaries of each article as a proxy for aspect annotation. We propose several straightforward baseline models for this task and conduct experiments on the dataset. Results highlight key challenges that existing summarization models face in this setting, such as proper pronoun handling of quoted sources and consistent explanation of time-sensitive events.
This paper introduces MediaSum, a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. To create this dataset, we collect interview transcripts from NPR and CNN and employ the overview and topic description
s as summaries. Compared with existing public corpora for dialogue summarization, our dataset is an order of magnitude larger and contains complex multi-party conversations from multiple domains. We conduct statistical analysis to demonstrate the unique positional bias exhibited in the transcripts of televised and radioed interviews. We also show that MediaSum can be used in transfer learning to improve a model's performance on other dialogue summarization tasks.
Recent advances in using retrieval components over external knowledge sources have shown impressive results for a variety of downstream tasks in natural language processing. Here, we explore the use of unstructured external knowledge sources of image
s and their corresponding captions for improving visual question answering (VQA). First, we train a novel alignment model for embedding images and captions in the same space, which achieves substantial improvement in performance on image-caption retrieval w.r.t. similar methods. Second, we show that retrieval-augmented multi-modal transformers using the trained alignment model improve results on VQA over strong baselines. We further conduct extensive experiments to establish the promise of this approach, and examine novel applications for inference time such as hot-swapping indices.
This paper introduces a new video-and-language dataset with human actions for multimodal logical inference, which focuses on intentional and aspectual expressions that describe dynamic human actions. The dataset consists of 200 videos, 5,554 action l
abels, and 1,942 action triplets of the form (subject, predicate, object) that can be easily translated into logical semantic representations. The dataset is expected to be useful for evaluating multimodal inference systems between videos and semantically complicated sentences including negation and quantification.
Recent development in NLP shows a strong trend towards refining pre-trained models with a domain-specific dataset. This is especially the case for response generation where emotion plays an important role. However, existing empathetic datasets remain
small, delaying research efforts in this area, for example, the development of emotion-aware chatbots. One main technical challenge has been the cost of manually annotating dialogues with the right emotion labels. In this paper, we describe a large-scale silver dataset consisting of 1M dialogues annotated with 32 fine-grained emotions, eight empathetic response intents, and the Neutral category. To achieve this goal, we have developed a novel data curation pipeline starting with a small seed of manually annotated data and eventually scaling it to a satisfactory size. We compare its quality against a state-of-the-art gold dataset using both offline experiments and visual validation methods. The resultant procedure can be used to create similar datasets in the same domain as well as in other domains.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
