في الآونة الأخيرة، حققت نماذج اللغة (LMS) أداء كبيرا في العديد من مهام NLU، التي حفزت اهتماما واسع النطاق للتطبيقات المحتملة في المجال العلمي والاجتماعي.ومع ذلك، واجهت LMS الكثير من الانتقاد لما إذا كانت قادرة حقا على التفكير في NLU.في هذا العمل، نقترح طريقة تشخيصية للمنطق المنطقي من الدرجة الأولى (FOL) مع معيار جديد مقترح، Logicnli.Logicnli عبارة عن مجموعة بيانات ذات طراز NLI الذي تم تنشيطه بشكل فعال من مستهدف فولت المستهدف من استنتاج المنطقي ويمكن استخدامه لتشخيص LMS من أربعة وجهات نظر: الدقة والمتانة والتعميم والتفسيرية.كشفت تجارب على بيرت وروبرتا و XLNET، عن نقاط الضعف في هؤلاء LMS بشأن التفكير فول، مما يحفز الاستكشاف المستقبلي لتعزيز قدرة المنطق.
Recently, language models (LMs) have achieved significant performance on many NLU tasks, which has spurred widespread interest for their possible applications in the scientific and social area. However, LMs have faced much criticism of whether they are truly capable of reasoning in NLU. In this work, we propose a diagnostic method for first-order logic (FOL) reasoning with a new proposed benchmark, LogicNLI. LogicNLI is an NLI-style dataset that effectively disentangles the target FOL reasoning from commonsense inference and can be used to diagnose LMs from four perspectives: accuracy, robustness, generalization, and interpretability. Experiments on BERT, RoBERTa, and XLNet, have uncovered the weaknesses of these LMs on FOL reasoning, which motivates future exploration to enhance the reasoning ability.
References used
https://aclanthology.org/
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel
First-order meta-learning algorithms have been widely used in practice to learn initial model parameters that can be quickly adapted to new tasks due to their efficiency and effectiveness. However, existing studies find that meta-learner can overfit
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence. Existing cognitive science literature on defeasible reasoning suggests that a person forms a mental model'' of the problem scenario
Since language is a natural concrete phenomenon, it became a fact that
language has been a matter of induction by making it go through
experiment in attempt to attain the rules that can take hold of the
language's partial phenomena and organize th
In modern natural language processing pipelines, it is common practice to pretrain'' a generative language model on a large corpus of text, and then to finetune'' the created representations by continuing to train them on a discriminative textual inf