Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDiagnosing the First-Order Logical Reasoning Ability Through LogicNLI
تشخيص القدرة المنطقية من الدرجة الأولى من خلال Logicnli
228
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في الآونة الأخيرة، حققت نماذج اللغة (LMS) أداء كبيرا في العديد من مهام NLU، التي حفزت اهتماما واسع النطاق للتطبيقات المحتملة في المجال العلمي والاجتماعي.ومع ذلك، واجهت LMS الكثير من الانتقاد لما إذا كانت قادرة حقا على التفكير في NLU.في هذا العمل، نقترح طريقة تشخيصية للمنطق المنطقي من الدرجة الأولى (FOL) مع معيار جديد مقترح، Logicnli.Logicnli عبارة عن مجموعة بيانات ذات طراز NLI الذي تم تنشيطه بشكل فعال من مستهدف فولت المستهدف من استنتاج المنطقي ويمكن استخدامه لتشخيص LMS من أربعة وجهات نظر: الدقة والمتانة والتعميم والتفسيرية.كشفت تجارب على بيرت وروبرتا و XLNET، عن نقاط الضعف في هؤلاء LMS بشأن التفكير فول، مما يحفز الاستكشاف المستقبلي لتعزيز قدرة المنطق.
Recently, language models (LMs) have achieved significant performance on many NLU tasks, which has spurred widespread interest for their possible applications in the scientific and social area. However, LMs have faced much criticism of whether they are truly capable of reasoning in NLU. In this work, we propose a diagnostic method for first-order logic (FOL) reasoning with a new proposed benchmark, LogicNLI. LogicNLI is an NLI-style dataset that effectively disentangles the target FOL reasoning from commonsense inference and can be used to diagnose LMs from four perspectives: accuracy, robustness, generalization, and interpretability. Experiments on BERT, RoBERTa, and XLNet, have uncovered the weaknesses of these LMs on FOL reasoning, which motivates future exploration to enhance the reasoning ability.
References used
https://aclanthology.org/
rate research
Read More
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel
RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with directly interpretable logical operators. Our experimental results show RL training with the proposed method converges significantly faster than other state-of-the-art neuro-symbolic methods in a TextWorld benchmark.
First-order meta-learning algorithms have been widely used in practice to learn initial model parameters that can be quickly adapted to new tasks due to their efficiency and effectiveness. However, existing studies find that meta-learner can overfit
to some specific adaptation when we have heterogeneous tasks, leading to significantly degraded performance. In Natural Language Processing (NLP) applications, datasets are often diverse and each task has its unique characteristics. Therefore, to address the overfitting issue when applying first-order meta-learning to NLP applications, we propose to reduce the variance of the gradient estimator used in task adaptation. To this end, we develop a variance-reduced first-order meta-learning algorithm. The core of our algorithm is to introduce a novel variance reduction term to the gradient estimation when performing the task adaptation. Experiments on two NLP applications: few-shot text classification and multi-domain dialog state tracking demonstrate the superior performance of our proposed method.
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence. Existing cognitive science literature on defeasible reasoning suggests that a person forms a mental model'' of the problem scenario
before answering questions. Our research goal asks whether neural models can similarly benefit from envisioning the question scenario before answering a defeasible query. Our approach is, given a question, to have a model first create a graph of relevant influences, and then leverage that graph as an additional input when answering the question. Our system, CURIOUS, achieves a new state-of-the-art on three different defeasible reasoning datasets. This result is significant as it illustrates that performance can be improved by guiding a system to think about'' a question and explicitly model the scenario, rather than answering reflexively.
Since language is a natural concrete phenomenon, it became a fact that
language has been a matter of induction by making it go through
experiment in attempt to attain the rules that can take hold of the
language's partial phenomena and organize th
em in general regulations,
and if we observed the linguistic matter which grammarians investigated,
we could find that their work involved both the complete induction and
the incomplete induction according to Aristotle's induction method, but
they disagreed with this method in accordance with the nature of Islamic
method of thinking; therefore they had their own distinguishing method
of induction.
In modern natural language processing pipelines, it is common practice to pretrain'' a generative language model on a large corpus of text, and then to finetune'' the created representations by continuing to train them on a discriminative textual inf
erence task. However, it is not immediately clear whether the logical meaning necessary to model logical entailment is captured by language models in this paradigm. We examine this pretrain-finetune recipe with language models trained on a synthetic propositional language entailment task, and present results on test sets probing models' knowledge of axioms of first order logic.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
