Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAccountable Error Characterization
توصيف خطأ المساءلة
42
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يطلب عملاء أنظمة التعلم الآلية المساءلة من الشركات التي توظف هذه الخوارزميات لمهام التنبؤ المختلفة. تتطلب المساءلة فهم حدود النظام وحالة التنبؤات الخاطئة، حيث غالبا ما يهتم العملاء بفهم التنبؤات غير الصحيحة، يتم امتصاص المطورين النموذجيين في العثور على طرق يمكن استخدامها للحصول على تحسينات تدريجية على نظام موجود. لذلك، نقترح طريقة توصيف خطأ مسؤولة، AEC، لفهم متى وحيث تحدث الأخطاء ضمن النماذج النسخة السوداء الموجودة. يسمح AEC، كما شيدت مع ميزات لغوية مفهومة للإنسان، المطورين النموذجيين لتحديد المصادر الرئيسية تلقائيا من الأخطاء لنظام تصنيف معين. يمكن استخدامه أيضا للعينة لمجموعة نقاط الإدخال الأكثر تفاعيمية في الجولة التالية من التدريب. نقوم بإجراء اكتشاف خطأ لمهمة تحليل المعنويات باستخدام AEC كدراسة حالة. تظهر نتائجنا على مهمة مشاعر العينة أن AEC قادر على تمييز التنبؤات الخاطئة في فئات غير قابلة للفطرة البشرية وتحقق أيضا نتائج واعدة على اختيار العينات الخاطئة بالمقارنة مع أخذ العينات القائمة على عدم اليقين.
Customers of machine learning systems demand accountability from the companies employing these algorithms for various prediction tasks. Accountability requires understanding of system limit and condition of erroneous predictions, as customers are often interested in understanding the incorrect predictions, and model developers are absorbed in finding methods that can be used to get incremental improvements to an existing system. Therefore, we propose an accountable error characterization method, AEC, to understand when and where errors occur within the existing black-box models. AEC, as constructed with human-understandable linguistic features, allows the model developers to automatically identify the main sources of errors for a given classification system. It can also be used to sample for the set of most informative input points for a next round of training. We perform error detection for a sentiment analysis task using AEC as a case study. Our results on the sample sentiment task show that AEC is able to characterize erroneous predictions into human understandable categories and also achieves promising results on selecting erroneous samples when compared with the uncertainty-based sampling.
References used
https://aclanthology.org/
rate research
Read More
GECko+ : a Grammatical and Discourse Error Correction Tool We introduce GECko+, a web-based writing assistance tool for English that corrects errors both at the sentence and at the discourse level. It is based on two state-of-the-art models for gramm
ar error correction and sentence ordering. GECko+ is available online as a web application that implements a pipeline combining the two models.
Although grammatical error correction (GEC) has achieved good performance on texts written by learners of English as a second language, performance on low error density domains where texts are written by English speakers of varying levels of proficie
ncy can still be improved. In this paper, we propose a contrastive learning approach to encourage the GEC model to assign a higher probability to a correct sentence while reducing the probability of incorrect sentences that the model tends to generate, so as to improve the accuracy of the model. Experimental results show that our approach significantly improves the performance of GEC models in low error density domains, when evaluated on the benchmark CWEB dataset.
This paper investigates how to correct Chinese text errors with types of mistaken, missing and redundant characters, which are common for Chinese native speakers. Most existing models based on detect-correct framework can correct mistaken characters,
but cannot handle missing or redundant characters due to inconsistency between model inputs and outputs. Although Seq2Seq-based or sequence tagging methods provide solutions to the three error types and achieved relatively good results in English context, they do not perform well in Chinese context according to our experiments. In our work, we propose a novel alignment-agnostic detect-correct framework that can handle both text aligned and non-aligned situations and can serve as a cold start model when no annotation data are provided. Experimental results on three datasets demonstrate that our method is effective and achieves a better performance than most recent published models.
The present research aims at synthesizing micro-porous and fluorescent polymer by microwave assisted synthesis allowing to reduce preparation time of 3 days at 180 °C using conventional method.
Quality Estimation (QE) for Machine Translation has been shown to reach relatively high accuracy in predicting sentence-level scores, relying on pretrained contextual embeddings and human-produced quality scores. However, the lack of explanations alo
ng with decisions made by end-to-end neural models makes the results difficult to interpret. Furthermore, word-level annotated datasets are rare due to the prohibitive effort required to perform this task, while they could provide interpretable signals in addition to sentence-level QE outputs. In this paper, we propose a novel QE architecture which tackles both the word-level data scarcity and the interpretability limitations of recent approaches. Sentence-level and word-level components are jointly pretrained through an attention mechanism based on synthetic data and a set of MT metrics embedded in a common space. Our approach is evaluated on the Eval4NLP 2021 shared task and our submissions reach the first position in all language pairs. The extraction of metric-to-input attention weights show that different metrics focus on different parts of the source and target text, providing strong rationales in the decision-making process of the QE model.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
