Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCOSMic: A Coherence-Aware Generation Metric for Image Descriptions
الكونية: متري تدوين التماسك من أوصاف الصورة
336
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعتمد مطورو نماذج جيل النص على مقاييس التقييم الآلي كمستقلة للتقييمات اليدوية البطيئة والمكلفة. ومع ذلك، كافحت مقاييس تقسيم الصور لإعطاء تقديرات مستفادة دقيقة للنجاح الدلالي والبراغماتي لنص الإخراج. نحن نتطلع إلى هذا الضعف عن طريق إدخال أول متري تعلم القيادة المستفادة لتقييم أوصاف الصورة. نهجنا مستوحى من النظريات الحاسوبية للتخلص من أهداف المعلومات باستخدام الاتساق. نقدم مجموعة بيانات من الصورة - وصف أزواج المشروح مع علاقات الاتساق. ثم قمنا بتدريب مقياس عمل متماسك على مجموعة فرعية من مجموعة بيانات التسميات المفاهيمية وقياس فعاليتها --- قدرتها على التنبؤ بالتصنيفات البشرية للتسميات التوضيحية الإنتاجية --- في مجموعة اختبار تتكون من صور خارج المجال. نوضح معامل ارتباط كيندل كيندل كيندل لتقسيطنا المقترح مع الأحكام الإنسانية لنتائج عدد من نماذج تواتير التسمية التوضيحية لمواصلة التماسك الواحد عند مقارنتها بالعديد من المقاييس الأخرى بما في ذلك المقاييس المستفادة المقترحة مؤخرا مثل bleurt و bertscore.
Developers of text generation models rely on automated evaluation metrics as a stand-in for slow and expensive manual evaluations. However, image captioning metrics have struggled to give accurate learned estimates of the semantic and pragmatic success of output text. We address this weakness by introducing the first discourse-aware learned generation metric for evaluating image descriptions. Our approach is inspired by computational theories of discourse for capturing information goals using coherence. We present a dataset of image--description pairs annotated with coherence relations. We then train a coherence-aware metric on a subset of the Conceptual Captions dataset and measure its effectiveness---its ability to predict human ratings of output captions---on a test set composed of out-of-domain images. We demonstrate a higher Kendall Correlation Coefficient for our proposed metric with the human judgments for the results of a number of state-of-the-art coherence-aware caption generation models when compared to several other metrics including recently proposed learned metrics such as BLEURT and BERTScore.
References used
https://aclanthology.org/
rate research
Read More
This work aims to explane and analysise some cosmic
particles that reach to the earth with super high energy
using the hypothesis of increasing the speed of light dated
to ancient past time ,based on the energy conservation law
and the mechanism
of transformation between particle state
and wave state.As example ,the speed of neutrin has been
taken for studying.
Abstract While pretrained language models (LMs) have driven impressive gains over morpho-syntactic and semantic tasks, their ability to model discourse and pragmatic phenomena is less clear. As a step towards a better understanding of their discourse
modeling capabilities, we propose a sentence intrusion detection task. We examine the performance of a broad range of pretrained LMs on this detection task for English. Lacking a dataset for the task, we introduce INSteD, a novel intruder sentence detection dataset, containing 170,000+ documents constructed from English Wikipedia and CNN news articles. Our experiments show that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity. Further results over a novel linguistic probe dataset show that there is substantial room for improvement, especially in the cross- domain setting.
Zero-shot cross-domain dialogue state tracking (DST) enables us to handle unseen domains without the expense of collecting in-domain data. In this paper, we propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST. Spec
ifically, our model first encodes a dialogue context and a slot with a pre-trained self-attentive encoder, and generates slot value in auto-regressive manner. In addition, we incorporate Slot Type Informed Descriptions that capture the shared information of different slots to facilitates the cross-domain knowledge transfer. Experimental results on MultiWOZ shows that our model significantly improve existing state-of-the-art results in zero-shot cross-domain setting.
Text generation from semantic graphs is traditionally performed with deterministic methods, which generate a unique description given an input graph. However, the generation problem admits a range of acceptable textual outputs, exhibiting lexical, sy
ntactic and semantic variation. To address this disconnect, we present two main contributions. First, we propose a stochastic graph-to-text model, incorporating a latent variable in an encoder-decoder model, and its use in an ensemble. Second, to assess the diversity of the generated sentences, we propose a new automatic evaluation metric which jointly evaluates output diversity and quality in a multi-reference setting. We evaluate the models on WebNLG datasets in English and Russian, and show an ensemble of stochastic models produces diverse sets of generated sentences while, retaining similar quality to state-of-the-art models.
We study the impact of using rich and diverse textual descriptions of classes for zero-shot learning (ZSL) on ImageNet. We create a new dataset ImageNet-Wiki that matches each ImageNet class to its corresponding Wikipedia article. We show that merely
employing these Wikipedia articles as class descriptions yields much higher ZSL performance than prior works. Even a simple model using this type of auxiliary data outperforms state-of-the-art models that rely on standard features of word embedding encodings of class names. These results highlight the usefulness and importance of textual descriptions for ZSL, as well as the relative importance of auxiliary data type compared to the algorithmic progress. Our experimental results also show that standard zero-shot learning approaches generalize poorly across categories of classes.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
