Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTextual Time Travel: A Temporally Informed Approach to Theory of Mind
السفر الوقت النصي: نهج مستنير مؤقتا لنظرية العقل
165
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يجب أن تكون أنظمة معالجة اللغة الطبيعية مثل وكلاء الحوار قادرة على سبب معتقدات الآخرين ونواياهم ورغباتهم. هذه القدرة، التي تسمى نظرية العقل (توم)، أمر بالغ الأهمية، حيث تتيح نموذج للتنبؤ وتفسير احتياجات المستخدمين بناء على حالاتهم العقلية. يقيم خط الأبحاث الحديث إمكانية توم من النماذج العصبية المعززة بالذاكرة الحالية من خلال الإجابة على السؤال. تؤدي هذه النماذج بشكل سيء على مهام الاعتقاد الكاذبة حيث تختلف المعتقدات عن الواقع، خاصة عندما تحتوي مجموعة البيانات على جمل مشتتة. في هذه الورقة، نقترح نهجا جديدا مستنرا مؤقتا لتحسين قدرة توم النماذج العصبية المعززة بالذاكرة. يتضمن نموذجنا بشعورا حول عقول الكيانات وتتبع حالاتهم العقلية لأنهم يتطورون بمرور الوقت من خلال مرور موسع. ثم يستجيب للاستعلامات من خلال السفر النصي - I.E.، عن طريق الوصول إلى الذاكرة المخزنة لخطوة زمنية سابقة. نقوم بتقييم نموذجنا على TOM Datasets ويجد أن هذا النهج يحسن الأداء، خاصة من خلال تصحيح الحالات الذهنية المتوقعة مطابقة الاعتقاد الخاطئ.
Natural language processing systems such as dialogue agents should be able to reason about other people's beliefs, intentions and desires. This capability, called theory of mind (ToM), is crucial, as it allows a model to predict and interpret the needs of users based on their mental states. A recent line of research evaluates the ToM capability of existing memory-augmented neural models through question-answering. These models perform poorly on false belief tasks where beliefs differ from reality, especially when the dataset contains distracting sentences. In this paper, we propose a new temporally informed approach for improving the ToM capability of memory-augmented neural models. Our model incorporates priors about the entities' minds and tracks their mental states as they evolve over time through an extended passage. It then responds to queries through textual time travel--i.e., by accessing the stored memory of an earlier time step. We evaluate our model on ToM datasets and find that this approach improves performance, particularly by correcting the predicted mental states to match the false belief.
References used
https://aclanthology.org/
rate research
Read More
An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collaboration and communication. To e
nable theory of mind modeling in situated interactions, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners' beliefs of the world and of each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for several theory of mind tasks.
Much of the progress in contemporary NLP has come from learning representations, such as masked language model (MLM) contextual embeddings, that turn challenging problems into simple classification tasks. But how do we quantify and explain this effec
t? We adapt general tools from computational learning theory to fit the specific characteristics of text datasets and present a method to evaluate the compatibility between representations and tasks. Even though many tasks can be easily solved with simple bag-of-words (BOW) representations, BOW does poorly on hard natural language inference tasks. For one such task we find that BOW cannot distinguish between real and randomized labelings, while pre-trained MLM representations show 72x greater distinction between real and random labelings than BOW. This method provides a calibrated, quantitative measure of the difficulty of a classification-based NLP task, enabling comparisons between representations without requiring empirical evaluations that may be sensitive to initializations and hyperparameters. The method provides a fresh perspective on the patterns in a dataset and the alignment of those patterns with specific labels.
We propose using a multilabel probing task to assess the morphosyntactic representations of multilingual word embeddings. This tweak on canonical probing makes it easy to explore morphosyntactic representations, both holistically and at the level of
individual features (e.g., gender, number, case), and leads more naturally to the study of how language models handle co-occurring features (e.g., agreement phenomena). We demonstrate this task with multilingual BERT (Devlin et al., 2018), training probes for seven typologically diverse languages: Afrikaans, Croatian, Finnish, Hebrew, Korean, Spanish, and Turkish. Through this simple but robust paradigm, we verify that multilingual BERT renders many morphosyntactic features simultaneously extractable. We further evaluate the probes on six held-out languages: Arabic, Chinese, Marathi, Slovenian, Tagalog, and Yoruba. This zero-shot style of probing has the added benefit of revealing which cross-linguistic properties a language model recognizes as being shared by multiple languages.
Adversarial examples expose the vulnerabilities of natural language processing (NLP) models, and can be used to evaluate and improve their robustness. Existing techniques of generating such examples are typically driven by local heuristic rules that
are agnostic to the context, often resulting in unnatural and ungrammatical outputs. This paper presents CLARE, a ContextuaLized AdversaRial Example generation model that produces fluent and grammatical outputs through a mask-then-infill procedure. CLARE builds on a pre-trained masked language model and modifies the inputs in a context-aware manner. We propose three contextualized perturbations, Replace, Insert and Merge, that allow for generating outputs of varied lengths. CLARE can flexibly combine these perturbations and apply them at any position in the inputs, and is thus able to attack the victim model more effectively with fewer edits. Extensive experiments and human evaluation demonstrate that CLARE outperforms the baselines in terms of attack success rate, textual similarity, fluency and grammaticality.
We consider the task of linking social media accounts that belong to the same author in an automated fashion on the basis of the content and meta-data of the corresponding document streams. We focus on learning an embedding that maps variable-sized s
amples of user activity--ranging from single posts to entire months of activity--to a vector space, where samples by the same author map to nearby points. Our approach does not require human-annotated data for training purposes, which allows us to leverage large amounts of social media content. The proposed model outperforms several competitive baselines under a novel evaluation framework modeled after established recognition benchmarks in other domains. Our method achieves high linking accuracy, even with small samples from accounts not seen at training time, a prerequisite for practical applications of the proposed linking framework.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
