Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUOR at SemEval-2021 Task 12: On Crowd Annotations; Learning with Disagreements to optimise crowd truth
UR في مهمة Semeval-2021 12: على التعليقات التوضيحية للحشد؛التعلم مع خلافات لتحسين الحقيقة الحشد
191
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تم استخدام Growdsourcing بشكل مجيئي لتعليق مجموعات هائلة من البيانات.ومع ذلك، فإن العقبات الرئيسية التي تحول دون استخدام ملصقات من مصادر الحموشة هي ضوضاء وأخطاء من التعليقات الشرحية غير الخبراء.في هذا العمل، يقترح مقارنتين تتعامل مع الضوضاء والأخطاء في ملصقات الحشد.يستخدم النهج الأول تقليل الحد الأدنى على علم الحدة (SAM)، وهي تقنية التحسين بقوة بالملصقات الصاخبة.ينفد النهج الآخر على أن طبقة شبكة عصبية تدعى SoftMax-Crowdlayer مصممة خصيصا للتعلم من التعليقات التوضيحية من الحشد.وفقا للنتائج، يمكن للنهج المقترحة تحسين أداء نموذج الشبكة المتبقية الواسعة ونموذج التصور متعدد الطبقات المطبقة على مجموعات بيانات المصادر في الحشد في مجال معالجة الصور.كما أنه يحتوي على نتائج مماثلة ومقارنة مع تقنية التصويت الأغلبية عند تطبيقها على مجال البيانات المتسلسل حيث يتم استخدام تمثيلات التشفير الثنائية من المحولات (Bert) كطراز أساسي في كلا الحالتين.
Crowdsourcing has been ubiquitously used for annotating enormous collections of data. However, the major obstacles to using crowd-sourced labels are noise and errors from non-expert annotations. In this work, two approaches dealing with the noise and errors in crowd-sourced labels are proposed. The first approach uses Sharpness-Aware Minimization (SAM), an optimization technique robust to noisy labels. The other approach leverages a neural network layer called softmax-Crowdlayer specifically designed to learn from crowd-sourced annotations. According to the results, the proposed approaches can improve the performance of the Wide Residual Network model and Multi-layer Perception model applied on crowd-sourced datasets in the image processing domain. It also has similar and comparable results with the majority voting technique when applied to the sequential data domain whereby the Bidirectional Encoder Representations from Transformers (BERT) is used as the base model in both instances.
References used
https://aclanthology.org/
rate research
Read More
Disagreement between coders is ubiquitous in virtually all datasets annotated with human judgements in both natural language processing and computer vision. However, most supervised machine learning methods assume that a single preferred interpretati
on exists for each item, which is at best an idealization. The aim of the SemEval-2021 shared task on learning with disagreements (Le-Wi-Di) was to provide a unified testing framework for methods for learning from data containing multiple and possibly contradictory annotations covering the best-known datasets containing information about disagreements for interpreting language and classifying images. In this paper we describe the shared task and its results.
This paper describes a system submitted by team BigGreen to LCP 2021 for predicting the lexical complexity of English words in a given context. We assemble a feature engineering-based model with a deep neural network model founded on BERT. While BERT
itself performs competitively, our feature engineering-based model helps in extreme cases, eg. separating instances of easy and neutral difficulty. Our handcrafted features comprise a breadth of lexical, semantic, syntactic, and novel phonological measures. Visualizations of BERT attention maps offer insight into potential features that Transformers models may learn when fine-tuned for lexical complexity prediction. Our ensembled predictions score reasonably well for the single word subtask, and we demonstrate how they can be harnessed to perform well on the multi word expression subtask too.
Among the tasks motivated by the proliferation of misinformation, propaganda detection is particularly challenging due to the deficit of fine-grained manual annotations required to train machine learning models. Here we show how data from other relat
ed tasks, including credibility assessment, can be leveraged in multi-task learning (MTL) framework to accelerate the training process. To that end, we design a BERT-based model with multiple output layers, train it in several MTL scenarios and perform evaluation against the SemEval gold standard.
Lexical complexity plays an important role in reading comprehension. lexical complexity prediction (LCP) can not only be used as a part of Lexical Simplification systems, but also as a stand-alone application to help people better reading. This paper
presents the winning system we submitted to the LCP Shared Task of SemEval 2021 that capable of dealing with both two subtasks. We first perform fine-tuning on numbers of pre-trained language models (PLMs) with various hyperparameters and different training strategies such as pseudo-labelling and data augmentation. Then an effective stacking mechanism is applied on top of the fine-tuned PLMs to obtain the final prediction. Experimental results on the Complex dataset show the validity of our method and we rank first and second for subtask 2 and 1.
Humor detection has become a topic of interest for several research teams, especially those involved in socio-psychological studies, with the aim to detect the humor and the temper of a targeted population (e.g. a community, a city, a country, the em
ployees of a given company). Most of the existing studies have formulated the humor detection problem as a binary classification task, whereas it revolves around learning the sense of humor by evaluating its different degrees. In this paper, we propose an end-to-end deep Multi-Task Learning (MTL) model to detect and rate humor and offense. It consists of a pre-trained transformer encoder and task-specific attention layers. The model is trained using MTL uncertainty loss weighting to adaptively combine all sub-tasks objective functions. Our MTL model tackles all sub-tasks of the SemEval-2021 Task-7 in one end-to-end deep learning system and shows very promising results.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
