Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCLIPScore: A Reference-free Evaluation Metric for Image Captioning
Clipscore: مقياس تقييم مجاني للمرجعية
290
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تم الاعتماد بشكل تقليدي في الصورة على التقييمات التلقائية القائمة على المرجع، حيث يتم مقارنة تعليق الآلات مع التسميات التوضيحية التي كتبها البشر. هذا على النقيض من الطريقة الخالية من المرجعية التي يقوم بها البشر بجودة التسمية التوضيحية. في هذه الورقة، نبلغ عن النتيجة التجريبية المدهشة التي كليب (Radford et al.، 2021)، نموذج متعدد الوسائط مسبقا على أزواج من صورة 400 متر + التسمية التوضيحية من الويب، يمكن استخدامه للتقييم التلقائي القوي لمعالجة الصور دون الحاجة للحصول على المراجع. تثبت التجارب التي تمتد من عدة شرجي أن قيصرنا الجديد المجاني، مقطورات، يحقق أعلى ارتباط بالأحكام البشرية، مما يتفوق على المقاييس القائمة على المرجع الحالي مثل عصير التفاح والتوابل. تثبت تجارب الحصول على المعلومات على أن Clipscore، مع تركيزها الضيق على توافق نص الصورة، هو تكميلية للمقاييس القائمة على مرجعية موجودة تؤكد على أوجه التشابه النصي النصي. وبالتالي، نقدم أيضا نسخة معدنية مرجعية، Refclipscore، والذي يحقق حتى ارتباط أعلى. تتجاوز مهام الوصف الحرفي، تكشف العديد من دراسات الحالة المجالات التي تنفذ فيها clipscore جيدا (صور Clip-Art، تصنيف النص)، ولكن أيضا حيث إنها أضعف نسبيا مقارنة بالمقاييس المستندة إلى المراجع، على سبيل المثال، التسميات التوضيحية الإخبارية التي تتطلب معرفة سياقية أكثر ثراء وبعد
Image captioning has conventionally relied on reference-based automatic evaluations, where machine captions are compared against captions written by humans. This is in contrast to the reference-free manner in which humans assess caption quality. In this paper, we report the surprising empirical finding that CLIP (Radford et al., 2021), a cross-modal model pretrained on 400M image+caption pairs from the web, can be used for robust automatic evaluation of image captioning without the need for references. Experiments spanning several corpora demonstrate that our new reference-free metric, CLIPScore, achieves the highest correlation with human judgements, outperforming existing reference-based metrics like CIDEr and SPICE. Information gain experiments demonstrate that CLIPScore, with its tight focus on image-text compatibility, is complementary to existing reference-based metrics that emphasize text-text similarities. Thus, we also present a reference-augmented version, RefCLIPScore, which achieves even higher correlation. Beyond literal description tasks, several case studies reveal domains where CLIPScore performs well (clip-art images, alt-text rating), but also where it is relatively weaker in comparison to reference-based metrics, e.g., news captions that require richer contextual knowledge.
References used
https://aclanthology.org/
rate research
Read More
Reference-free evaluation has the potential to make machine translation evaluation substantially more scalable, allowing us to pivot easily to new languages or domains. It has been recently shown that the probabilities given by a large, multilingual
model can achieve state of the art results when used as a reference-free metric. We experiment with various modifications to this model, and demonstrate that by scaling it up we can match the performance of BLEU. We analyze various potential weaknesses of the approach, and find that it is surprisingly robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities.
In image captioning, multiple captions are often provided as ground truths, since a valid caption is not always uniquely determined. Conventional methods randomly select a single caption and treat it as correct, but there have been few effective trai
ning methods that utilize multiple given captions. In this paper, we proposed two training technique for making effective use of multiple reference captions: 1) validity-based caption sampling (VBCS), which prioritizes the use of captions that are estimated to be highly valid during training, and 2) weighted caption smoothing (WCS), which applies smoothing only to the relevant words the reference caption to reflect multiple reference captions simultaneously. Experiments show that our proposed methods improve CIDEr by 2.6 points and BLEU4 by 0.9 points from baseline on the MSCOCO dataset.
Many modern machine translation evaluation metrics like BERTScore, BLEURT, COMET, MonoTransquest or XMoverScore are based on black-box language models. Hence, it is difficult to explain why these metrics return certain scores. This year's Eval4NLP sh
ared task tackles this challenge by searching for methods that can extract feature importance scores that correlate well with human word-level error annotations. In this paper we show that unsupervised metrics that are based on tokenmatching can intrinsically provide such scores. The submitted system interprets the similarities of the contextualized word-embeddings that are used to compute (X)BERTScore as word-level importance scores.
We present an information retrieval-based question answer system to answer legal questions. The system is not limited to a predefined set of questions or patterns and uses both sparse vector search and embeddings for input to a BERT-based answer re-r
anking system. A combination of general domain and legal domain data is used for training. This natural question answering system is in production and is used commercially.
بناء نظام ذكي يقوم بالتعرف على الأصناف الموجودة في صورة وتوليد توصيف نصي لهذه الأغراض الموجودة في الصورة.
استخدمنا الشبكات العصبونية الملتفة Convolutional Neural Networks للقيام بعملية استخلاص الأصناف الموجودة في الصورة، وأدخلنا هذه الأصناف إلى شبكة
عصبونية تكرارية Recurrent Neural Network للقيام بعملية توليد التوصيف النصي.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
