Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTopic Transferable Table Question Answering
موضوع الجدول القابل للتحويل الإجابة
284
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
حققت نماذج الإجابة على الجدول (TableQa) ضعيفا (TableQA) أداء حديثة من خلال استخدام محول بيرت المدرب مسبقا إلى ترميز سؤال وجداول لإنتاج استعلام منظم للسؤال. ومع ذلك، في الإعدادات العملية يتم نشر أنظمة Tableqa عبر جدول كوربورا وجود توزيعات موضوعية وتوزيعات كلمة متميزة تماما من Bertraining Corpus. في هذا العمل، نحاكي سيناريو التحول العملي من خلال تصميم معايير التحدي الجديدة Wikisql-TS و WiKiTe-TS، وتتألف من تقسيم اختبار قطار Dev في خمس مجموعات موضوع مميزة، استنادا إلى مجموعات بيانات Wikisql والأسئلة الشائعة. نوضح تجريبيا أنه على الرغم من التدريب المسبق على نص واسع النطاق، يتحلل أداء النماذج بشكل كبير عندما يتم تقييمها على مواضيع غير مرئية. ردا على ذلك، نقترح T3QA (موضوع الإجابة على الجدول القابل للتحويل) إطار التكيف العملي ل TableQA يتألف من: (1) حقن المفردات المحددة للموضوع في بيرت، (2) مولد محول نص إلى نص جديد (مثل T5، GPT2) يركز خط أنابيب توليد السؤال الطبيعي المستندة إلى اللغة الطبيعية على توليد بيانات التدريب الخاصة بالموضوع، و (3) Reveer نموذج منطقي. نظهر أن T3QA يوفر خط الأساس الجيد بشكل معقول لمعايير تحول الموضوع لدينا. نعتقد أن معاييرنا المنفصلة لدينا ستؤدي إلى حلول طاولة قوية مناسبة للنشر العملي
Weakly-supervised table question-answering (TableQA) models have achieved state-of-art performance by using pre-trained BERT transformer to jointly encoding a question and a table to produce structured query for the question. However, in practical settings TableQA systems are deployed over table corpora having topic and word distributions quite distinct from BERT's pretraining corpus. In this work we simulate the practical topic shift scenario by designing novel challenge benchmarks WikiSQL-TS and WikiTable-TS, consisting of train-dev-test splits in five distinct topic groups, based on the popular WikiSQL and WikiTable-Questions datasets. We empirically show that, despite pre-training on large open-domain text, performance of models degrades significantly when they are evaluated on unseen topics. In response, we propose T3QA (Topic Transferable Table Question Answering) a pragmatic adaptation framework for TableQA comprising of: (1) topic-specific vocabulary injection into BERT, (2) a novel text-to-text transformer generator (such as T5, GPT2) based natural language question generation pipeline focused on generating topic-specific training data, and (3) a logical form re-ranker. We show that T3QA provides a reasonably good baseline for our topic shift benchmarks. We believe our topic split benchmarks will lead to robust TableQA solutions that are better suited for practical deployment
References used
https://aclanthology.org/
rate research
Read More
Many open-domain question answering problems can be cast as a textual entailment task, where a question and candidate answers are concatenated to form hypotheses. A QA system then determines if the supporting knowledge bases, regarded as potential pr
emises, entail the hypotheses. In this paper, we investigate a neural-symbolic QA approach that integrates natural logic reasoning within deep learning architectures, towards developing effective and yet explainable question answering models. The proposed model gradually bridges a hypothesis and candidate premises following natural logic inference steps to build proof paths. Entailment scores between the acquired intermediate hypotheses and candidate premises are measured to determine if a premise entails the hypothesis. As the natural logic reasoning process forms a tree-like, hierarchical structure, we embed hypotheses and premises in a Hyperbolic space rather than Euclidean space to acquire more precise representations. Empirically, our method outperforms prior work on answering multiple-choice science questions, achieving the best results on two publicly available datasets. The natural logic inference process inherently provides evidence to help explain the prediction process.
While diverse question answering (QA) datasets have been proposed and contributed significantly to the development of deep learning models for QA tasks, the existing datasets fall short in two aspects. First, we lack QA datasets covering complex ques
tions that involve answers as well as the reasoning processes to get them. As a result, the state-of-the-art QA research on numerical reasoning still focuses on simple calculations and does not provide the mathematical expressions or evidence justifying the answers. Second, the QA community has contributed a lot of effort to improve the interpretability of QA models. However, they fail to explicitly show the reasoning process, such as the evidence order for reasoning and the interactions between different pieces of evidence. To address the above shortcoming, we introduce NOAHQA, a conversational and bilingual QA dataset with questions requiring numerical reasoning with compound mathematical expressions. With NOAHQA, we develop an interpretable reasoning graph as well as the appropriate evaluation metric to measure the answer quality. We evaluate the state-of-the-art QA models trained using existing QA datasets on NOAHQA and show that the best among them can only achieve 55.5 exact match scores, while the human performance is 89.7. We also present a new QA model for generating a reasoning graph where the reasoning graph metric still has a large gap compared with that of humans, eg, 28 scores.
The evaluation of question answering models compares ground-truth annotations with model predictions. However, as of today, this comparison is mostly lexical-based and therefore misses out on answers that have no lexical overlap but are still semanti
cally similar, thus treating correct answers as false. This underestimation of the true performance of models hinders user acceptance in applications and complicates a fair comparison of different models. Therefore, there is a need for an evaluation metric that is based on semantics instead of pure string similarity. In this short paper, we present SAS, a cross-encoder-based metric for the estimation of semantic answer similarity, and compare it to seven existing metrics. To this end, we create an English and a German three-way annotated evaluation dataset containing pairs of answers along with human judgment of their semantic similarity, which we release along with an implementation of the SAS metric and the experiments. We find that semantic similarity metrics based on recent transformer models correlate much better with human judgment than traditional lexical similarity metrics on our two newly created datasets and one dataset from related work.
NLP research in Hebrew has largely focused on morphology and syntax, where rich annotated datasets in the spirit of Universal Dependencies are available. Semantic datasets, however, are in short supply, hindering crucial advances in the development o
f NLP technology in Hebrew. In this work, we present ParaShoot, the first question answering dataset in modern Hebrew. The dataset follows the format and crowdsourcing methodology of SQuAD, and contains approximately 3000 annotated examples, similar to other question-answering datasets in low-resource languages. We provide the first baseline results using recently-released BERT-style models for Hebrew, showing that there is significant room for improvement on this task.
We present an information retrieval-based question answer system to answer legal questions. The system is not limited to a predefined set of questions or patterns and uses both sparse vector search and embeddings for input to a BERT-based answer re-r
anking system. A combination of general domain and legal domain data is used for training. This natural question answering system is in production and is used commercially.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
