Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAre NLP Models really able to Solve Simple Math Word Problems?
هي نماذج NLP قادرة حقا على حل مشاكل كلمة الرياضيات البسيطة؟
385
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
شهدت مشكلة تصميم حلول NLP لمشاكل كلمة الرياضيات (MWP) نشاط بحثي مستمر ومكاسب ثابتة في دقة الاختبار. نظرا لأن الحلفل الموجودين يحققون أداء عاليا على مجموعات البيانات القياسية للمستوى الابتدائي الذي يحتوي على مشاكل في الكلمات الحسابية المجهولة الأولى، إلا أن هذه المشكلات غالبا ما يتم حلها في كثير من الأحيان "مع الجزء الأكبر من الاهتمام بالبحث إلى MWPS أكثر تعقيدا. في هذه الورقة، قصرنا انتباهنا إلى اللغة الإنجليزية MWPs تدرس في الصفوف الأربعة والأقل. نحن نقدم دليلا قويا على أن وحدات MWP الحالية تعتمد على الاستدلال الضحلة لتحقيق أداء عال في مجموعات البيانات القياسية. تحقيقا لهذه الغاية، نظهر أن حفلات MWP التي لا تملك الوصول إلى السؤال المطلوب في MWP، لا يزال بإمكانك حل جزء كبير من MWPS. وبالمثل، فإن النماذج التي تعالج mwps كحقيبة من الكلمات يمكن أن تحقق أيضا دقة عالية بشكل مدهش. علاوة على ذلك، نقدم مجموعة بيانات تحدي، Svamp، تم إنشاؤها من خلال تطبيق الاختلافات المختارة بعناية على الأمثلة التي تم أخذ عينات منها من مجموعات البيانات الحالية. إن أفضل الدقة التي تحققت بها النماذج الحديثة أقل بكثير على Svamp، وبالتالي إظهار الكثير لا يزال يتعين القيام به حتى لأبسط MWPS.
The problem of designing NLP solvers for math word problems (MWP) has seen sustained research activity and steady gains in the test accuracy. Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered solved'' with the bulk of research attention moving to more complex MWPs. In this paper, we restrict our attention to English MWPs taught in grades four and lower. We provide strong evidence that the existing MWP solvers rely on shallow heuristics to achieve high performance on the benchmark datasets. To this end, we show that MWP solvers that do not have access to the question asked in the MWP can still solve a large fraction of MWPs. Similarly, models that treat MWPs as bag-of-words can also achieve surprisingly high accuracy. Further, we introduce a challenge dataset, SVAMP, created by applying carefully chosen variations over examples sampled from existing datasets. The best accuracy achieved by state-of-the-art models is substantially lower on SVAMP, thus showing that much remains to be done even for the simplest of the MWPs.
References used
https://aclanthology.org/
rate research
Read More
Current neural math solvers learn to incorporate commonsense or domain knowledge by utilizing pre-specified constants or formulas. However, as these constants and formulas are mainly human-specified, the generalizability of the solvers is limited. In
this paper, we propose to explicitly retrieve the required knowledge from math problemdatasets. In this way, we can determinedly characterize the required knowledge andimprove the explainability of solvers. Our two algorithms take the problem text andthe solution equations as input. Then, they try to deduce the required commonsense and domain knowledge by integrating information from both parts. We construct two math datasets and show the effectiveness of our algorithms that they can retrieve the required knowledge for problem-solving.
In this article, we tackle the math word problem, namely, automatically answering a mathematical problem according to its textual description. Although recent methods have demonstrated their promising results, most of these methods are based on templ
ate-based generation scheme which results in limited generalization capability. To this end, we propose a novel human-like analogical learning method in a recall and learn manner. Our proposed framework is composed of modules of memory, representation, analogy, and reasoning, which are designed to make a new exercise by referring to the exercises learned in the past. Specifically, given a math word problem, the model first retrieves similar questions by a memory module and then encodes the unsolved problem and each retrieved question using a representation module. Moreover, to solve the problem in a way of analogy, an analogy module and a reasoning module with a copy mechanism are proposed to model the interrelationship between the problem and each retrieved question. Extensive experiments on two well-known datasets show the superiority of our proposed algorithm as compared to other state-of-the-art competitors from both overall performance comparison and micro-scope studies.
While solving math word problems automatically has received considerable attention in the NLP community, few works have addressed probability word problems specifically. In this paper, we employ and analyse various neural models for answering such wo
rd problems. In a two-step approach, the problem text is first mapped to a formal representation in a declarative language using a sequence-to-sequence model, and then the resulting representation is executed using a probabilistic programming system to provide the answer. Our best performing model incorporates general-domain contextualised word representations that were finetuned using transfer learning on another in-domain dataset. We also apply end-to-end models to this task, which bring out the importance of the two-step approach in obtaining correct solutions to probability problems.
Recent studies have revealed a security threat to natural language processing (NLP) models, called the Backdoor Attack. Victim models can maintain competitive performance on clean samples while behaving abnormally on samples with a specific trigger w
ord inserted. Previous backdoor attacking methods usually assume that attackers have a certain degree of data knowledge, either the dataset which users would use or proxy datasets for a similar task, for implementing the data poisoning procedure. However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples. Experimental results on sentiment analysis and sentence-pair classification tasks show that our method is more efficient and stealthier. We hope this work can raise the awareness of such a critical security risk hidden in the embedding layers of NLP models. Our code is available at https://github.com/lancopku/Embedding-Poisoning.
Research in NLP has mainly focused on factoid questions, with the goal of finding quick and reliable ways of matching a query to an answer. However, human discourse involves more than that: it contains non-canonical questions deployed to achieve spec
ific communicative goals. In this paper, we investigate this under-studied aspect of NLP by introducing a targeted task, creating an appropriate corpus for the task and providing baseline models of diverse nature. With this, we are also able to generate useful insights on the task and open the way for future research in this direction.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
