Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userNDH-Full: Learning and Evaluating Navigational Agents on Full-Length Dialogue
NDH-Full: تعلم وتقييم العوامل الملاحية على حوار كامل الطول
271
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يتماشى التواصل بين الوكلاء البشري والهاتف المحمول بشكل متزايد حيث يتم نشر هذه الوكلاء على نطاق واسع في حياتنا اليومية. الرؤية والحوار الملاحة هي واحدة من المهام التي تقوم بتقييم قدرة الوكيل على التفاعل مع البشر للحصول على المساعدة والتنقل على أساس ردود اللغة الطبيعية. في هذه الورقة، نستكشف الملاحة من مهمة تاريخ الحوار (NDH)، والتي تستند إلى مجموعة بيانات الملاحة في الرؤية والحوار التعاوني (CVDN)، وتقديم نموذج أحدث من الفن الذي تم بناؤه عند الرؤية محولات اللغة. ومع ذلك، على الرغم من تحقيق الأداء التنافسي، نجد أن الوكيل في مهمة NDH لم يتم تقييمه بشكل مناسب من خلال التقدم المتقدي الرئيسي - الهدف. من خلال تحليل عدم تطابق الأداء بين تقدم المرمى ومقاييس أخرى (على سبيل المثال، تزييف الوقت الديناميكي الطبيعي) من نموذج الحديث لدينا، نوضح أن إعداد المهام المستندة إلى المسار الفرعي NDH (أي، التنقل إلى مسار جزئي بناء على مراسله لا توفر مجموعة فرعية من الحوار الكامل الوكيل مع إشارة إشراف كافية نحو منطقة الهدف. لذلك، نقترح إعداد مهمة جديدة يسمى NDH - الكامل الذي يأخذ الحوار الكامل ومسار التنقل بأكمله كحل واحد. نقدم نموذجا أساسيا قويا وإظهار النتائج الأولية في هذه المهمة الجديدة. وصفنا كذلك العديد من الأساليب التي نحاولها، من أجل تحسين الأداء النموذجي (بناء على تعلم المناهج الدراسية، ما قبل التدريب، وتعزيز البيانات)، مما يشير إلى طرق تدريب مفيدة محتملة في هذه المهمة الجديدة NDH الجديدة.
Communication between human and mobile agents is getting increasingly important as such agents are widely deployed in our daily lives. Vision-and-Dialogue Navigation is one of the tasks that evaluate the agent's ability to interact with humans for assistance and navigate based on natural language responses. In this paper, we explore the Navigation from Dialogue History (NDH) task, which is based on the Cooperative Vision-and-Dialogue Navigation (CVDN) dataset, and present a state-of-the-art model which is built upon Vision-Language transformers. However, despite achieving competitive performance, we find that the agent in the NDH task is not evaluated appropriately by the primary metric -- Goal Progress. By analyzing the performance mismatch between Goal Progress and other metrics (e.g., normalized Dynamic Time Warping) from our state-of-the-art model, we show that NDH's sub-path based task setup (i.e., navigating partial trajectory based on its correspondent subset of the full dialogue) does not provide the agent with enough supervision signal towards the goal region. Therefore, we propose a new task setup called NDH-Full which takes the full dialogue and the whole navigation path as one instance. We present a strong baseline model and show initial results on this new task. We further describe several approaches that we try, in order to improve the model performance (based on curriculum learning, pre-training, and data-augmentation), suggesting potential useful training methods on this new NDH-Full task.
References used
https://aclanthology.org/
rate research
Read More
Enabling open-domain dialogue systems to ask clarifying questions when appropriate is an important direction for improving the quality of the system response. Namely, for cases when a user request is not specific enough for a conversation system to p
rovide an answer right away, it is desirable to ask a clarifying question to increase the chances of retrieving a satisfying answer. To address the problem of asking clarifying questions in open-domain dialogues': (1) we collect and release a new dataset focused on open-domain single- and multi-turn conversations, (2) we benchmark several state-of-the-art neural baselines, and (3) we propose a pipeline consisting of offline and online steps for evaluating the quality of clarifying questions in various dialogues. These contributions are suitable as a foundation for further research.
Intelligent agents that are confronted with novel concepts in situated environments will need to ask their human teammates questions to learn about the physical world. To better understand this problem, we need data about asking questions in situated
task-based interactions. To this end, we present the Human-Robot Dialogue Learning (HuRDL) Corpus - a novel dialogue corpus collected in an online interactive virtual environment in which human participants play the role of a robot performing a collaborative tool-organization task. We describe the corpus data and a corresponding annotation scheme to offer insight into the form and content of questions that humans ask to facilitate learning in a situated environment. We provide the corpus as an empirically-grounded resource for improving question generation in situated intelligent agents.
Conversational agents trained on large unlabeled corpora of human interactions will learn patterns and mimic behaviors therein, which include offensive or otherwise toxic behavior. We introduce a new human-and-model-in-the-loop framework for evaluati
ng the toxicity of such models, and compare a variety of existing methods in both the cases of non-adversarial and adversarial users that expose their weaknesses. We then go on to propose two novel methods for safe conversational agents, by either training on data from our new human-and-model-in-the-loop framework in a two-stage system, or ''baking-in'' safety to the generative model itself. We find our new techniques are (i) safer than existing models; while (ii) maintaining usability metrics such as engagingness relative to state-of-the-art chatbots. In contrast, we expose serious safety issues in existing standard systems like GPT2, DialoGPT, and BlenderBot.
We study the task of learning and evaluating Chinese idiom embeddings. We first construct a new evaluation dataset that contains idiom synonyms and antonyms. Observing that existing Chinese word embedding methods may not be suitable for learning idio
m embeddings, we further present a BERT-based method that directly learns embedding vectors for individual idioms. We empirically compare representative existing methods and our method. We find that our method substantially outperforms existing methods on the evaluation dataset we have constructed.
This research traces, after conducting a wide literature survey, the areas not covered by prominent agent oriented software engineering (AOSE) methodologies. Each methodology has its strength and weakness and focuses on some stages of software
devel
opment lifecycle but not all stages. This paper presents an addition to a well established AOSE methodology (MaSE). MaSE is considered one of the strongest in the field, it does not, however, support handling early requirements. This work integrates MaSE with another methodology known for its strength in early requirement representation. The integration implied the development of a wide set of translation rules between two different environments of notations and graphical representations. A software tool was developed to automate the translation and a case study is used to demonstrate the work.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
