Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userJournalistic Guidelines Aware News Image Captioning
المبادئ التوجيهية الصحفية تدرك أخبار الصورة التوضيحية
215
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
مهمة تقصير صورة المقال الإخباري يهدف إلى توليد تعليق وصفية وغنية بالمعلومات لصور المقالة الإخبارية.على عكس التسميات التوضيحية التقليدية التي تصف ببساطة محتوى الصورة بمصطلحات عامة، تتبع تعليق الصور الإخبارية إرشادات صحفية وتعتمد اعتمادا كبيرا على الكيانات المسماة لوصف محتوى الصورة، غالبا ما يرسم السياق من المقالة بأكملها.في هذا العمل، نقترح نهجا جديدا لهذه المهمة، بدافع من إرشادات التسمية التوضيحية التي يتبعها الصحفيون.نهجنا، المبادئ التوجيهية الصحفية تدرك صورة أخبار التسمية التوضيحية (Joganic)، ترفد بنية التسميات التوضيحية لتحسين جودة الجيل وتوجيه تصميم التمثيل الخاص بنا.النتائج التجريبية، بما في ذلك دراسات التفصيل التفصيلية، على مجموعة من مجموعات بيانات واسعة النطاق للجمهورية على نطاق واسع أن جوجاني يتفوق بشكل كبير على الأساليب الحديثة على حد سواء على جيل التسمية التوضيحية ومقاييس الكيانية المسمى.
The task of news article image captioning aims to generate descriptive and informative captions for news article images. Unlike conventional image captions that simply describe the content of the image in general terms, news image captions follow journalistic guidelines and rely heavily on named entities to describe the image content, often drawing context from the whole article they are associated with. In this work, we propose a new approach to this task, motivated by caption guidelines that journalists follow. Our approach, Journalistic Guidelines Aware News Image Captioning (JoGANIC), leverages the structure of captions to improve the generation quality and guide our representation design. Experimental results, including detailed ablation studies, on two large-scale publicly available datasets show that JoGANIC substantially outperforms state-of-the-art methods both on caption generation and named entity related metrics.
References used
https://aclanthology.org/
rate research
Read More
We propose Visual News Captioner, an entity-aware model for the task of news image captioning. We also introduce Visual News, a large-scale benchmark consisting of more than one million news images along with associated news articles, image captions,
author information, and other metadata. Unlike the standard image captioning task, news images depict situations where people, locations, and events are of paramount importance. Our proposed method can effectively combine visual and textual features to generate captions with richer information such as events and entities. More specifically, built upon the Transformer architecture, our model is further equipped with novel multi-modal feature fusion techniques and attention mechanisms, which are designed to generate named entities more accurately. Our method utilizes much fewer parameters while achieving slightly better prediction results than competing methods. Our larger and more diverse Visual News dataset further highlights the remaining challenges in captioning news images.
Modern web content - news articles, blog posts, educational resources, marketing brochures - is predominantly multimodal. A notable trait is the inclusion of media such as images placed at meaningful locations within a textual narrative. Most often,
such images are accompanied by captions - either factual or stylistic (humorous, metaphorical, etc.) - making the narrative more engaging to the reader. While standalone image captioning has been extensively studied, captioning an image based on external knowledge such as its surrounding text remains under-explored. In this paper, we study this new task: given an image and an associated unstructured knowledge snippet, the goal is to generate a contextual caption for the image.
بناء نظام ذكي يقوم بالتعرف على الأصناف الموجودة في صورة وتوليد توصيف نصي لهذه الأغراض الموجودة في الصورة.
استخدمنا الشبكات العصبونية الملتفة Convolutional Neural Networks للقيام بعملية استخلاص الأصناف الموجودة في الصورة، وأدخلنا هذه الأصناف إلى شبكة
عصبونية تكرارية Recurrent Neural Network للقيام بعملية توليد التوصيف النصي.
Developers of text generation models rely on automated evaluation metrics as a stand-in for slow and expensive manual evaluations. However, image captioning metrics have struggled to give accurate learned estimates of the semantic and pragmatic succe
ss of output text. We address this weakness by introducing the first discourse-aware learned generation metric for evaluating image descriptions. Our approach is inspired by computational theories of discourse for capturing information goals using coherence. We present a dataset of image--description pairs annotated with coherence relations. We then train a coherence-aware metric on a subset of the Conceptual Captions dataset and measure its effectiveness---its ability to predict human ratings of output captions---on a test set composed of out-of-domain images. We demonstrate a higher Kendall Correlation Coefficient for our proposed metric with the human judgments for the results of a number of state-of-the-art coherence-aware caption generation models when compared to several other metrics including recently proposed learned metrics such as BLEURT and BERTScore.
In this paper, we aim to address the challenges surrounding the translation of ancient Chinese text: (1) The linguistic gap due to the difference in eras results in translations that are poor in quality, and (2) most translations are missing the cont
extual information that is often very crucial to understanding the text. To this end, we improve upon past translation techniques by proposing the following: We reframe the task as a multi-label prediction task where the model predicts both the translation and its particular era. We observe that this helps to bridge the linguistic gap as chronological context is also used as auxiliary information. We validate our framework on a parallel corpus annotated with chronology information and show experimentally its efficacy in producing quality translation outputs. We release both the code and the data for future research.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
