Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTowards the Application of Calibrated Transformers to the Unsupervised Estimation of Question Difficulty from Text
نحو تطبيق المحولات المعايرة إلى تقدير غير مدهش للصعوبة من النص
250
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
أن تكون قادرا على أداء تقدير صعوبة الأسئلة بدقة (QDE) تحسين دقة تقييم الطلاب وتحسين تجربة التعلم. الأساليب التقليدية إلى QDE هي إما ذاتية أو إدخال تأخير طويل قبل أن يتم استخدام أسئلة جديدة لتقييم الطلاب. وبالتالي، اقترح العمل الأخير النهج القائم على التعلم في التعلم للتغلب على هذه القيود. يستخدمون أسئلة من صعوبة معروفة لتدريب النماذج القادرة على استنتاج صعوبة الأسئلة من نصها. بمجرد التدريب، يمكن استخدامها لأداء QDE من الأسئلة التي تم إنشاؤها حديثا. توفر الأساليب الحالية النماذج الخاضعة للإشراف والتي تعتمد على نطاق ومتطلبات مجموعة بيانات كبيرة من الأسئلة المعروفة للتدريب. لذلك، لا يمكن استخدامها إذا كانت هذه البيانات غير متوفرة (للدورات الجديدة على منصة التعلم الإلكتروني). في هذا العمل، نقوم بتجربة إمكانية أداء QDE من النص بطريقة غير منشأة. على وجه التحديد، نستخدم عدم اليقين من الأسئلة المعايرة في الرد النماذج كوكيل للصعوبة المتصورة للإنسان. تظهر تجاربنا نتائج واعدة، مما يشير إلى أن عدم اليقين النموذجي يمكن الاستفادة بنجاح لأداء QDE من النص، مما يقلل من كل من التكاليف والوقت المنقضي.
Being able to accurately perform Question Difficulty Estimation (QDE) can improve the accuracy of students' assessment and better their learning experience. Traditional approaches to QDE are either subjective or introduce a long delay before new questions can be used to assess students. Thus, recent work proposed machine learning-based approaches to overcome these limitations. They use questions of known difficulty to train models capable of inferring the difficulty of questions from their text. Once trained, they can be used to perform QDE of newly created questions. Existing approaches employ supervised models which are domain-dependent and require a large dataset of questions of known difficulty for training. Therefore, they cannot be used if such a dataset is not available ( for new courses on an e-learning platform). In this work, we experiment with the possibility of performing QDE from text in an unsupervised manner. Specifically, we use the uncertainty of calibrated question answering models as a proxy of human-perceived difficulty. Our experiments show promising results, suggesting that model uncertainty could be successfully leveraged to perform QDE from text, reducing both costs and elapsed time.
References used
https://aclanthology.org/
rate research
Read More
The style transfer task (here style is used in a broad authorial'' sense with many aspects including register, sentence structure, and vocabulary choice) takes text input and rewrites it in a specified target style preserving the meaning, but alterin
g the style of the source text to match that of the target. Much of the existing research on this task depends on the use of parallel datasets. In this work we employ recent results in unsupervised cross-lingual language modeling (XLM) and machine translation to effect style transfer while treating the input data as unaligned. First, we show that adding content embeddings'' to the XLM which capture human-specified groupings of subject matter can improve performance over the baseline model. Evaluation of style transfer has often relied on metrics designed for machine translation which have received criticism of their suitability for this task. As a second contribution, we propose the use of a suite of classical stylometrics as a useful complement for evaluation. We select a few such measures and include these in the analysis of our results.
This research shows the concept of sentence syntax and the text
syntax and the difference between them, beside their respective
areas .It also tries to specify the obstacles which prevent the
progress of this kind of linguistic lesson in our Arabi
an collages
.Then it stops at the trends of linguistic studies where such kind of
linguistic lesson appears .Also tries to monitor the reality of this
lingual lesson in the Syrian collages through one sample ,that is Al
Baath University .Finally finishes by the most important
recommendations which can contribute in developing this kind of
lingual lesson .
Weakly-supervised text classification aims to induce text classifiers from only a few user-provided seed words. The vast majority of previous work assumes high-quality seed words are given. However, the expert-annotated seed words are sometimes non-t
rivial to come up with. Furthermore, in the weakly-supervised learning setting, we do not have any labeled document to measure the seed words' efficacy, making the seed word selection process a walk in the dark''. In this work, we remove the need for expert-curated seed words by first mining (noisy) candidate seed words associated with the category names. We then train interim models with individual candidate seed words. Lastly, we estimate the interim models' error rate in an unsupervised manner. The seed words that yield the lowest estimated error rates are added to the final seed word set. A comprehensive evaluation of six binary classification tasks on four popular datasets demonstrates that the proposed method outperforms a baseline using only category name seed words and obtained comparable performance as a counterpart using expert-annotated seed words.
We propose a new reference-free summary quality evaluation measure, with emphasis on the faithfulness. The measure is based on finding and counting all probable potential inconsistencies of the summary with respect to the source document. The propose
d ESTIME, Estimator of Summary-to-Text Inconsistency by Mismatched Embeddings, correlates with expert scores in summary-level SummEval dataset stronger than other common evaluation measures not only in Consistency but also in Fluency. We also introduce a method of generating subtle factual errors in human summaries. We show that ESTIME is more sensitive to subtle errors than other common evaluation measures.
Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a range of downstream tasks. In this paper we explore whether it is possible to learn disentangl
ed representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspects. Concretely, we learn binary masks over transformer weights or hidden units to uncover subsets of features that correlate with a specific factor of variation; this eliminates the need to train a disentangled model from scratch for a particular task. We evaluate this method with respect to its ability to disentangle representations of sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. By combining masking with magnitude pruning we find that we can identify sparse subnetworks within BERT that strongly encode particular aspects (e.g., semantics) while only weakly encoding others (e.g., syntax). Moreover, despite only learning masks, disentanglement-via-masking performs as well as --- and often better than ---previously proposed methods based on variational autoencoders and adversarial training.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
