Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userParameter Space Factorization for Zero-Shot Learning across Tasks and Languages
المعلمة تعامل الفضاء للتعلم صفر النار عبر المهام واللغات
538
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
مجردة معظم مجموعات مهام NLP والأصناف اللغوية تفتقر إلى أمثلة في المجال للتدريب الخاضع للإشراف بسبب قلة البيانات المشروحة. كيف يمكن النماذج العصبية أن تجعل تعميمات فعالة للعينة من مجموعات لغات المهام مع البيانات المتاحة للموارد المنخفضة؟ في هذا العمل، نقترح نموذجا إيلائيا بايزيا لمساحة المعلمات العصبية. نفترض أن هذه المساحة يمكن أن تعصبها في متغيرات كامنة لكل لغة وكل مهمة. نحن نستنتج المشتريات حول هذه المتغيرات الكامنة بناء على بيانات من مجموعات لغة المهام المشاهدة من خلال الاستدلال المتغيرات. وهذا يتيح تصنيف صفري بالرصاص على مجموعات غير مرئية في وقت التنبؤ. على سبيل المثال، نظرا لبيانات التدريب للتعرف على الكيان المسمى (NER) في الفيتنامية ولليزة جزء من الكلام (POS) (POS) في Wolof، يمكن أن يؤدي نموذجنا إلى إجراء تنبؤات دقيقة ل NER في Wolof. على وجه الخصوص، نقوم بتجربة عينة متنوعة من 33 لغة من 4 قارات و 11 أسرة، وإظهار أن نموذجنا ينتج عنه نتائج قابلة للمقارنة أو أفضل من أساليب التحويل المتبادلة الصفرية من بين الفن. يتوفر الكود الخاص بنا في github.com/cambridgeltl/parameter-factorization.
Abstract Most combinations of NLP tasks and language varieties lack in-domain examples for supervised training because of the paucity of annotated data. How can neural models make sample-efficient generalizations from task--language combinations with available data to low-resource ones? In this work, we propose a Bayesian generative model for the space of neural parameters. We assume that this space can be factorized into latent variables for each language and each task. We infer the posteriors over such latent variables based on data from seen task--language combinations through variational inference. This enables zero-shot classification on unseen combinations at prediction time. For instance, given training data for named entity recognition (NER) in Vietnamese and for part-of-speech (POS) tagging in Wolof, our model can perform accurate predictions for NER in Wolof. In particular, we experiment with a typologically diverse sample of 33 languages from 4 continents and 11 families, and show that our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods. Our code is available at github.com/cambridgeltl/parameter-factorization.
References used
https://aclanthology.org/
rate research
Read More
We address the sampling bias and outlier issues in few-shot learning for event detection, a subtask of information extraction. We propose to model the relations between training tasks in episodic few-shot learning by introducing cross-task prototypes
. We further propose to enforce prediction consistency among classifiers across tasks to make the model more robust to outliers. Our extensive experiment shows a consistent improvement on three few-shot learning datasets. The findings suggest that our model is more robust when labeled data of novel event types is limited. The source code is available at http://github.com/laiviet/fsl-proact.
In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework. We propose an episode-based cross-attention (EpiCA) network which combines merits of cross-attention mechanism a
nd episode-based training strategy to recognize novel compositional concepts. Firstly, EpiCA bases on cross-attention to correlate conceptvisual information and utilizes the gated pooling layer to build contextualized representations for both images and concepts. The updated representations are used for a more indepth multi-modal relevance calculation for concept recognition. Secondly, a two-phase episode training strategy, especially the ransductive phase, is adopted to utilize unlabeled test examples to alleviate the low-resource learning problem. Experiments on two widelyused zero-shot compositional learning (ZSCL) benchmarks have demonstrated the effectiveness of the model compared with recent approaches on both conventional and generalized ZSCL settings.
Building automatic technical support system is an important yet challenge task. Conceptually, to answer a user question on a technical forum, a human expert has to first retrieve relevant documents, and then read them carefully to identify the answer
snippet. Despite huge success the researchers have achieved in coping with general domain question answering (QA), much less attentions have been paid for investigating technical QA. Specifically, existing methods suffer from several unique challenges (i) the question and answer rarely overlaps substantially and (ii) very limited data size. In this paper, we propose a novel framework of deep transfer learning to effectively address technical QA across tasks and domains. To this end, we present an adjustable joint learning approach for document retrieval and reading comprehension tasks. Our experiments on the TechQA demonstrates superior performance compared with state-of-the-art methods.
This paper describes our submission to SemEval 2021 Task 2. We compare XLM-RoBERTa Base and Large in the few-shot and zero-shot settings and additionally test the effectiveness of using a k-nearest neighbors classifier in the few-shot setting instead
of the more traditional multi-layered perceptron. Our experiments on both the multi-lingual and cross-lingual data show that XLM-RoBERTa Large, unlike the Base version, seems to be able to more effectively transfer learning in a few-shot setting and that the k-nearest neighbors classifier is indeed a more powerful classifier than a multi-layered perceptron when used in few-shot learning.
Stance detection on social media can help to identify and understand slanted news or commentary in everyday life. In this work, we propose a new model for zero-shot stance detection on Twitter that uses adversarial learning to generalize across topic
s. Our model achieves state-of-the-art performance on a number of unseen test topics with minimal computational costs. In addition, we extend zero-shot stance detection to topics not previously considered, highlighting future directions for zero-shot transfer.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
