Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userExample-Driven Intent Prediction with Observers
على سبيل المثال - تنبؤ النية بالمراقبين
239
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
التحدي الرئيسي في أبحاث أنظمة الحوار هو التكيف بشكل فعال وكفاءة مع مجالات جديدة. يتطلب نموذجا قابل للتطوير للتكيف تطوير النماذج التعميمية التي تؤدي بشكل جيد في إعدادات قليلة. في هذه الورقة، نركز على مشكلة تصنيف النية التي تهدف إلى تحديد نوايا المستخدمين المعطاة الكلام الموجهة إلى نظام الحوار. نقترح اقترابين لتحسين تعميم نماذج تصنيف الكلام: (1) مراقبون و (2) تدريب على سبيل المثال لقد أظهر العمل السابق أن النماذج التي تشبه بيرت تميل إلى تنسيق مبلغ كبير من الاهتمام ل [CLS] الرمز المميز، والتي نفترض النتائج في تمثيلات مخففة. المراقبون هم الرموز التي لا تحضرها، وهي بديل من رمزية [CLS] كتمثيل دلالي للكلمات. يتعلم التدريب على سبيل المثال أن تصنف الكلام من خلال مقارنة بالأمثلة، وبالتالي استخدام التشفير الأساسي كنموذج تشابه الجملة. هذه الأساليب مكملة؛ إن تحسين التمثيل من خلال المراقبين يسمحون بالنموذج الذي يحركه المثال إلى تحسين أوجه تشابه الجملة. عند دمجها، فإن الأساليب المقترحة تحقق نتائج أحدث نتائج من ثلاث مجموعات من مجموعات بيانات التنبؤ النية (Banking77، CLINC150، HWU64) في كلا البيانات الكاملة وإعدادات قليلة (10 أمثلة لكل نية). علاوة على ذلك، نوضح أن النهج المقترح يمكن أن ينقل إلى النوايا الجديدة وعبر مجموعات البيانات دون أي تدريب إضافي.
A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. These methods are complementary; improving the representation through observers allows the example-driven model to better measure sentence similarities. When combined, the proposed methods attain state-of-the-art results on three intent prediction datasets (banking77, clinc150, hwu64) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.
References used
https://aclanthology.org/
rate research
Read More
Intent detection is a key component of modern goal-oriented dialog systems that accomplish a user task by predicting the intent of users' text input. There are three primary challenges in designing robust and accurate intent detection models. First,
typical intent detection models require a large amount of labeled data to achieve high accuracy. Unfortunately, in practical scenarios it is more common to find small, unbalanced, and noisy datasets. Secondly, even with large training data, the intent detection models can see a different distribution of test data when being deployed in the real world, leading to poor accuracy. Finally, a practical intent detection model must be computationally efficient in both training and single query inference so that it can be used continuously and re-trained frequently. We benchmark intent detection methods on a variety of datasets. Our results show that Watson Assistant's intent detection model outperforms other commercial solutions and is comparable to large pretrained language models while requiring only a fraction of computational resources and training data. Watson Assistant demonstrates a higher degree of robustness when the training and test distributions differ.
In order to alleviate the huge demand for annotated datasets for different tasks, many recent natural language processing datasets have adopted automated pipelines for fast-tracking usable data. However, model training with such datasets poses a chal
lenge because popular optimization objectives are not robust to label noise induced in the annotation generation process. Several noise-robust losses have been proposed and evaluated on tasks in computer vision, but they generally use a single dataset-wise hyperparamter to control the strength of noise resistance. This work proposes novel instance-adaptive training frameworks to change single dataset-wise hyperparameters of noise resistance in such losses to be instance-wise. Such instance-wise noise resistance hyperparameters are predicted by special instance-level label quality predictors, which are trained along with the main classification models. Experiments on noisy and corrupted NLP datasets show that proposed instance-adaptive training frameworks help increase the noise-robustness provided by such losses, promoting the use of the frameworks and associated losses in NLP models trained with noisy data.
A private learning scheme TextHide was recently proposed to protect the private text data during the training phase via so-called instance encoding. We propose a novel reconstruction attack to break TextHide by recovering the private training data, a
nd thus unveil the privacy risks of instance encoding. We have experimentally validated the effectiveness of the reconstruction attack with two commonly-used datasets for sentence classification. Our attack would advance the development of privacy preserving machine learning in the context of natural language processing.
Knowledge Graphs (KGs) have become increasingly popular in the recent years. However, as knowledge constantly grows and changes, it is inevitable to extend existing KGs with entities that emerged or became relevant to the scope of the KG after its cr
eation. Research on updating KGs typically relies on extracting named entities and relations from text. However, these approaches cannot infer entities or relations that were not explicitly stated. Alternatively, embedding models exploit implicit structural regularities to predict missing relations, but cannot predict missing entities. In this article, we introduce a novel method to enrich a KG with new entities given their textual description. Our method leverages joint embedding models, hence does not require entities or relations to be named explicitly. We show that our approach can identify new concepts in a document corpus and transfer them into the KG, and we find that the performance of our method improves substantially when extended with techniques from association rule mining, text mining, and active learning.
We present a systematic study on multilingual and cross-lingual intent detection (ID) from spoken data. The study leverages a new resource put forth in this work, termed MInDS-14, a first training and evaluation resource for the ID task with spoken d
ata. It covers 14 intents extracted from a commercial system in the e-banking domain, associated with spoken examples in 14 diverse language varieties. Our key results indicate that combining machine translation models with state-of-the-art multilingual sentence encoders (e.g., LaBSE) yield strong intent detectors in the majority of target languages covered in MInDS-14, and offer comparative analyses across different axes: e.g., translation direction, impact of speech recognition, data augmentation from a related domain. We see this work as an important step towards more inclusive development and evaluation of multilingual ID from spoken data, hopefully in a much wider spectrum of languages compared to prior work.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
