Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTable-based Fact Verification With Salience-aware Learning
التحقق من الحقائق القائم على الطاولة مع التعلم العلمي
259
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
توفر الجداول معرفة قيمة يمكن استخدامها للتحقق من العبارات النصية. في حين أن عددا من الأعمال قد نظر في التحقق من الحقائق القائم على الطاولة، فإن المحاذاة المباشرة للبيانات الجذابية مع الرموز في البيانات النصية نادرا ما توفرها. علاوة على ذلك، فإن تدريب نموذج التحقق من الحقائق المعممة يتطلب بيانات تدريبية ملصقة وفيرة. في هذه الورقة، نقترح نظام رواية لمعالجة هذه المشكلات. مستوحاة من السببية المتعددة، يحدد نظامنا من رجال الصمغ على مستوى الرمز في البيان مع تقدير البحار الذي يستند إلى التحقيق. يتيح تقدير Salience التعلم المعزز للتحقق من الحقائق من وجهات نظر. من منظور واحد، يقوم نظامنا بإجراء تنبؤ ممثن بالبرنامج المريح لتعزيز النموذج للمحاذاة والتفكير بين الطاولة والبيان. من المنظور الآخر، ينطبق نظامنا على توضيح تكبير البيانات الإدراك بالاستثناء لإنشاء مجموعة متنوعة من مثيلات التدريب عن طريق استبدال المصطلحات غير البارزة. تظهر النتائج التجريبية على Tabract التحسن الفعال من خلال تقنيات التعلم التي أدركها Carience المقترحة، مما يؤدي إلى أداء Sota الجديد على المعيار.
Tables provide valuable knowledge that can be used to verify textual statements. While a number of works have considered table-based fact verification, direct alignments of tabular data with tokens in textual statements are rarely available. Moreover, training a generalized fact verification model requires abundant labeled training data. In this paper, we propose a novel system to address these problems. Inspired by counterfactual causality, our system identifies token-level salience in the statement with probing-based salience estimation. Salience estimation allows enhanced learning of fact verification from two perspectives. From one perspective, our system conducts masked salient token prediction to enhance the model for alignment and reasoning between the table and the statement. From the other perspective, our system applies salience-aware data augmentation to generate a more diverse set of training instances by replacing non-salient terms. Experimental results on TabFact show the effective improvement by the proposed salience-aware learning techniques, leading to the new SOTA performance on the benchmark.
References used
https://aclanthology.org/
rate research
Read More
Table-based fact verification task aims to verify whether the given statement is supported by the given semi-structured table. Symbolic reasoning with logical operations plays a crucial role in this task. Existing methods leverage programs that conta
in rich logical information to enhance the verification process. However, due to the lack of fully supervised signals in the program generation process, spurious programs can be derived and employed, which leads to the inability of the model to catch helpful logical operations. To address the aforementioned problems, in this work, we formulate the table-based fact verification task as an evidence retrieval and reasoning framework, proposing the Logic-level Evidence Retrieval and Graph-based Verification network (LERGV). Specifically, we first retrieve logic-level program-like evidence from the given table and statement as supplementary evidence for the table. After that, we construct a logic-level graph to capture the logical relations between entities and functions in the retrieved evidence, and design a graph-based verification network to perform logic-level graph-based reasoning based on the constructed graph to classify the final entailment relation. Experimental results on the large-scale benchmark TABFACT show the effectiveness of the proposed approach.
In this paper, we propose a novel fact checking and verification system to check claims against Wikipedia content. Our system retrieves relevant Wikipedia pages using Anserini, uses BERT-large-cased question answering model to select correct evidence
, and verifies claims using XLNET natural language inference model by comparing it with the evidence. Table cell evidence is obtained through looking for entity-matching cell values and TAPAS table question answering model. The pipeline utilizes zero-shot capabilities of existing models and all the models used in the pipeline requires no additional training. Our system got a FEVEROUS score of 0.06 and a label accuracy of 0.39 in FEVEROUS challenge.
This work describes the adaptation of a pretrained sequence-to-sequence model to the task of scientific claim verification in the biomedical domain. We propose a system called VerT5erini that exploits T5 for abstract retrieval, sentence selection, an
d label prediction, which are three critical sub-tasks of claim verification. We evaluate our pipeline on SciFACT, a newly curated dataset that requires models to not just predict the veracity of claims but also provide relevant sentences from a corpus of scientific literature that support the prediction. Empirically, our system outperforms a strong baseline in each of the three sub-tasks. We further show VerT5erini's ability to generalize to two new datasets of COVID-19 claims using evidence from the CORD-19 corpus.
Summarization evaluation remains an open research problem: current metrics such as ROUGE are known to be limited and to correlate poorly with human judgments. To alleviate this issue, recent work has proposed evaluation metrics which rely on question
answering models to assess whether a summary contains all the relevant information in its source document. Though promising, the proposed approaches have so far failed to correlate better than ROUGE with human judgments. In this paper, we extend previous approaches and propose a unified framework, named QuestEval. In contrast to established metrics such as ROUGE or BERTScore, QuestEval does not require any ground-truth reference. Nonetheless, QuestEval substantially improves the correlation with human judgments over four evaluation dimensions (consistency, coherence, fluency, and relevance), as shown in extensive experiments.
Understanding tables is an important and relevant task that involves understanding table structure as well as being able to compare and contrast information within cells. In this paper, we address this challenge by presenting a new dataset and tasks
that addresses this goal in a shared task in SemEval 2020 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS). Our dataset contains 981 manually-generated tables and an auto-generated dataset of 1980 tables providing over 180K statement and over 16M evidence annotations. SEM-TAB-FACTS featured two sub-tasks. In sub-task A, the goal was to determine if a statement is supported, refuted or unknown in relation to a table. In sub-task B, the focus was on identifying the specific cells of a table that provide evidence for the statement. 69 teams signed up to participate in the task with 19 successful submissions to subtask A and 12 successful submissions to subtask B. We present our results and main findings from the competition.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
