Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userGame-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index
اختيار المفردات المفروضة في اللعبة عبر قيمة shemley وفهرس البنزاف
120
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
المفردات المدخلات والتمثيلات المستفادة أمر حاسم لأداء نماذج NLP العصبية. يؤدي استخدام نتائج المفردات الكاملة إلى نماذج أقل وتفسير الذاكرة، مع طبقة التضمين غالبا ما تشكل غالبية المعلمات النموذجية. من الشائع، من الشائع استخدام مفردات أصغر لخفض متطلبات الذاكرة وبناء المزيد من النماذج الأربعة. نقترح طريقة اختيار المفردات التي تعرض الكلمات كأعضاء في فريق يحاول زيادة أداء النموذج إلى الحد الأقصى. نحن نطبق مؤشرات الطاقة من نظرية الألعاب التعاونية، بما في ذلك فهرس قيمة Shemley و Banzhaf، والتي تقيس الأهمية النسبية لأعضاء الفريق الفردي في إنجاز مهمة مشتركة. نحسب تقريبا هذه المؤشرات لتحديد الكلمات الأكثر نفوذا. يفحص تقييم التجريبي الخاص بنا مهام متعددة الوزراء، بما في ذلك الجملة والتصنيف المستندات، والرد على السؤال والتتبع النصي. نقارن مع خطوط الأساسيات التي تختار الكلمات القائمة على ترددات التردد و TF-IDF والانحدار بموجب منتظم L1، وإظهار أن اختيار المفردات النظري للعبة هذه تفوق كل خط الأساس على مجموعة من المهام والعملات المختلفة.
The input vocabulary and the representations learned are crucial to the performance of neural NLP models. Using the full vocabulary results in less explainable and more memory intensive models, with the embedding layer often constituting the majority of model parameters. It is thus common to use a smaller vocabulary to lower memory requirements and construct more interpertable models. We propose a vocabulary selection method that views words as members of a team trying to maximize the model's performance. We apply power indices from cooperative game theory, including the Shapley value and Banzhaf index, that measure the relative importance of individual team members in accomplishing a joint task. We approximately compute these indices to identify the most influential words. Our empirical evaluation examines multiple NLP tasks, including sentence and document classification, question answering and textual entailment. We compare to baselines that select words based on frequency, TF-IDF and regression coefficients under L1 regularization, and show that this game-theoretic vocabulary selection outperforms all baseline on a range of different tasks and datasets.
References used
https://aclanthology.org/
rate research
Read More
We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation. Our approach is suitable f
or large-scale datasets, applies to distant languages with unseen scripts, incurs only minor degradation on the translation performance for the original language pairs and provides competitive performance even in the case where we only possess monolingual data for the new languages.
We study the problem of generating counterfactual text for a classifier as a means for understanding and debugging classification. Given a textual input and a classification model, we aim to minimally alter the text to change the model's prediction.
White-box approaches have been successfully applied to similar problems in vision where one can directly optimize the continuous input. Optimization-based approaches become difficult in the language domain due to the discrete nature of text. We bypass this issue by directly optimizing in the latent space and leveraging a language model to generate candidate modifications from optimized latent representations. We additionally use Shapley values to estimate the combinatoric effect of multiple changes. We then use these estimates to guide a beam search for the final counterfactual text. We achieve favorable performance compared to recent white-box and black-box baselines using human and automatic evaluations. Ablation studies show that both latent optimization and the use of Shapley values improve success rate and the quality of the generated counterfactuals.
Transformers that are pre-trained on multilingual corpora, such as, mBERT and XLM-RoBERTa, have achieved impressive cross-lingual transfer capabilities. In the zero-shot transfer setting, only English training data is used, and the fine-tuned model i
s evaluated on another target language. While this works surprisingly well, substantial variance has been observed in target language performance between different fine-tuning runs, and in the zero-shot setup, no target-language development data is available to select among multiple fine-tuned models. Prior work has relied on English dev data to select among models that are fine-tuned with different learning rates, number of steps and other hyperparameters, often resulting in suboptimal choices. In this paper, we show that it is possible to select consistently better models when small amounts of annotated data are available in auxiliary pivot languages. We propose a machine learning approach to model selection that uses the fine-tuned model's own internal representations to predict its cross-lingual capabilities. In extensive experiments we find that this method consistently selects better models than English validation data across twenty five languages (including eight low-resource languages), and often achieves results that are comparable to model selection using target language development data.
The aim of vocabulary inventory prediction is to predict a learner's whole vocabulary based on a limited sample of query words. This paper approaches the problem starting from the 2-parameter Item Response Theory (IRT) model, giving each word in the
vocabulary a difficulty and discrimination parameter. The discrimination parameter is evaluated on the sub-problem of question item selection, familiar from the fields of Computerised Adaptive Testing (CAT) and active learning. Next, the effect of the discrimination parameter on prediction performance is examined, both in a binary classification setting, and in an information retrieval setting. Performance is compared with baselines based on word frequency. A number of different generalisation scenarios are examined, including generalising word difficulty and discrimination using word embeddings with a predictor network and testing on out-of-dataset data.
Transformer-based neural networks offer very good classification performance across a wide range of domains, but do not provide explanations of their predictions. While several explanation methods, including SHAP, address the problem of interpreting
deep learning models, they are not adapted to operate on state-of-the-art transformer-based neural networks such as BERT. Another shortcoming of these methods is that their visualization of explanations in the form of lists of most relevant words does not take into account the sequential and structurally dependent nature of text. This paper proposes the TransSHAP method that adapts SHAP to transformer models including BERT-based text classifiers. It advances SHAP visualizations by showing explanations in a sequential manner, assessed by human evaluators as competitive to state-of-the-art solutions.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
