Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userInterpreting Text Classifiers by Learning Context-sensitive Influence of Words
ترجمة نصوص النص عن طريق تعلم التأثير الحساس للسياق للكلمات
130
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تركز العديد من النهج الحالية لتفسير نماذج تصنيف النص على توفير الأهمية عشرات لأجزاء من نص الإدخال، مثل الكلمات، ولكن دون أي طريقة لاختبار أو تحسين طريقة التفسير نفسها. هذا له تأثير مزعج مشكلة فهم أو بناء الثقة في النموذج، مع طريقة التفسير نفسها إضافة إلى عتامة النموذج. علاوة على ذلك، فإن العشرات الأهمية حول الأمثلة الفردية عادة ما تكون لا تكفي لتوفير صورة كافية من السلوك النموذجي. لمعالجة هذه المخاوف، نقترح Moxie (تأثير النمذجة الحساسة للكلمات) بهدف تمكين واجهة أكثر ثراء للمستخدم للتفاعل مع النموذج الذي يتم تفسيره وإنتاج تنبؤات قابلة للإصابة. على وجه الخصوص، نهدف إلى تقديم تنبؤات لعشرات الأهمية والمعدات المضادة والتحيزات المستفادة مع Moxie. بالإضافة إلى ذلك، مع هدف التعلم العالمي، يوفر Moxie مسارا واضحا لاختبار وتحسين نفسها. نقيم موثوقية وكفاءة Moxie على مهمة تحليل المعنويات.
Many existing approaches for interpreting text classification models focus on providing importance scores for parts of the input text, such as words, but without a way to test or improve the interpretation method itself. This has the effect of compounding the problem of understanding or building trust in the model, with the interpretation method itself adding to the opacity of the model. Further, importance scores on individual examples are usually not enough to provide a sufficient picture of model behavior. To address these concerns, we propose MOXIE (MOdeling conteXt-sensitive InfluencE of words) with an aim to enable a richer interface for a user to interact with the model being interpreted and to produce testable predictions. In particular, we aim to make predictions for importance scores, counterfactuals and learned biases with MOXIE. In addition, with a global learning objective, MOXIE provides a clear path for testing and improving itself. We evaluate the reliability and efficiency of MOXIE on the task of sentiment analysis.
References used
https://aclanthology.org/
rate research
Read More
Abstract Phonological generalizations are finite-state. While Optimality Theory is a popular framework for modeling phonology, it is known to generate non-finite-state mappings and languages. This paper demonstrates that Optimality Theory is capable
of generating non-context-free languages, contributing to the characterization of its generative capacity. This is achieved with minimal modification to the theory as it is standardly employed.
We propose to tackle data-to-text generation tasks by directly splicing together retrieved segments of text from neighbor'' source-target pairs. Unlike recent work that conditions on retrieved neighbors but generates text token-by-token, left-to-righ
t, we learn a policy that directly manipulates segments of neighbor text, by inserting or replacing them in partially constructed generations. Standard techniques for training such a policy require an oracle derivation for each generation, and we prove that finding the shortest such derivation can be reduced to parsing under a particular weighted context-free grammar. We find that policies learned in this way perform on par with strong baselines in terms of automatic and human evaluation, but allow for more interpretable and controllable generation.
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is based on finding and counting all probable potential inconsistencies of the summary with respect to the source document. The propose
d ESTIME, Estimator of Summary-to-Text Inconsistency by Mismatched Embeddings, correlates with expert scores in summary-level SummEval dataset stronger than other common evaluation measures not only in Consistency but also in Fluency. We also introduce a method of generating subtle factual errors in human summaries. We show that ESTIME is more sensitive to subtle errors than other common evaluation measures.
As large-scale, pre-trained language models achieve human-level and superhuman accuracy on existing language understanding tasks, statistical bias in benchmark data and probing studies have recently called into question their true capabilities. For a
more informative evaluation than accuracy on text classification tasks can offer, we propose evaluating systems through a novel measure of prediction coherence. We apply our framework to two existing language understanding benchmarks with different properties to demonstrate its versatility. Our experimental results show that this evaluation framework, although simple in ideas and implementation, is a quick, effective, and versatile measure to provide insight into the coherence of machines' predictions.
Meta-learning has achieved great success in leveraging the historical learned knowledge to facilitate the learning process of the new task. However, merely learning the knowledge from the historical tasks, adopted by current meta-learning algorithms,
may not generalize well to testing tasks when they are not well-supported by training tasks. This paper studies a low-resource text classification problem and bridges the gap between meta-training and meta-testing tasks by leveraging the external knowledge bases. Specifically, we propose KGML to introduce additional representation for each sentence learned from the extracted sentence-specific knowledge graph. The extensive experiments on three datasets demonstrate the effectiveness of KGML under both supervised adaptation and unsupervised adaptation settings.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
