Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userOverview of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments
نظرة عامة على Germeval 2021 المشتركة مهمة حول تحديد تعليقات السامة والمشاركة، والحقائق
311
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
نقدم Germeval 2021 المهمة المشتركة بشأن تحديد تعليقات السامة والإخراجية والحقائق. تضم هذه المهمة المشتركة ثلاث مجموعات فرعية تصنيف ثنائية بهدف تحديد: تعليقات سامة، وإشراك التعليقات، والتعليقات التي تشمل مؤشرات على الحاجة إلى فحص الحقائق، يشار إليها هنا كتعليقات مطالبة بالحقائق. بناء على المهام المشتركة السابقة للجراثيم السابقة بشأن تحديد اللغة الهجومية في عامي 2018 و 2019، نقوم بتوسيع تعريف المهمة لهذا العام لتلبية طلب المشرفين ومديري المجتمع على إبراز التعليقات التي تعزز التواصل المحدد، وتشجيع المناقشات المتعمقة، وتشجيع المناقشات المتعمقة، و تحقق من الحقائق التي تعتمد خطوط الحجج. تشتمل مجموعة البيانات على 4،188 وظيفة مستخرجة من صفحة الفيسبوك لمظهر حديث سياسي ألماني من مذيع تلفزيوني عام وطني. الإطار النظري واختبارات الموثوقية الإضافية خلال عملية شروح البيانات ضمان جودة البيانات عالية بشكل خاص. كانت المهمة المشتركة 15 فريقا مشاركا تقدم 31 أشواط للمشاركة في التعليقات الفرعية بشأن التعليقات السامة، و 25 أشواطا للمتعلقة الفرعية حول إشراك التعليقات، و 31 مقابل الترابط الفرعي على تعليقات المطالبة بالحقائق. يمكن العثور على موقع المهام المشترك في https://germeval2021toxic.github.io/sharedtask/.
We present the GermEval 2021 shared task on the identification of toxic, engaging, and fact-claiming comments. This shared task comprises three binary classification subtasks with the goal to identify: toxic comments, engaging comments, and comments that include indications of a need for fact-checking, here referred to as fact-claiming comments. Building on the two previous GermEval shared tasks on the identification of offensive language in 2018 and 2019, we extend this year's task definition to meet the demand of moderators and community managers to also highlight comments that foster respectful communication, encourage in-depth discussions, and check facts that lines of arguments rely on. The dataset comprises 4,188 posts extracted from the Facebook page of a German political talk show of a national public television broadcaster. A theoretical framework and additional reliability tests during the data annotation process ensure particularly high data quality. The shared task had 15 participating teams submitting 31 runs for the subtask on toxic comments, 25 runs for the subtask on engaging comments, and 31 for the subtask on fact-claiming comments. The shared task website can be found at https://germeval2021toxic.github.io/SharedTask/.
References used
https://aclanthology.org/
rate research
Read More
In this paper, we report on our approach to addressing the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments for the German language. We submitted three runs for each subtask based on ensembles of three mo
dels each using contextual embeddings from pre-trained language models using SVM and neural-network-based classifiers. We include language-specific as well as language-agnostic language models -- both with and without fine-tuning. We observe that for the runs we submitted that the SVM models overfitted the training data and this affected the aggregation method (simple majority voting) of the ensembles. The model records a lower performance on the test set than on the training set. Exploring the issue of overfitting we uncovered that due to a bug in the pipeline the runs we submitted had not been trained on the full set but only on a small training set. Therefore in this paper we also include the results we get when trained on the full training set which demonstrate the power of ensembles.
The availability of language representations learned by large pretrained neural network models (such as BERT and ELECTRA) has led to improvements in many downstream Natural Language Processing tasks in recent years. Pretrained models usually differ i
n pretraining objectives, architectures, and datasets they are trained on which can affect downstream performance. In this contribution, we fine-tuned German BERT and German ELECTRA models to identify toxic (subtask 1), engaging (subtask 2), and fact-claiming comments (subtask 3) in Facebook data provided by the GermEval 2021 competition. We created ensembles of these models and investigated whether and how classification performance depends on the number of ensemble members and their composition. On out-of-sample data, our best ensemble achieved a macro-F1 score of 0.73 (for all subtasks), and F1 scores of 0.72, 0.70, and 0.76 for subtasks 1, 2, and 3, respectively.
In this paper we present UPAppliedCL's contribution to the GermEval 2021 Shared Task. In particular, we participated in Subtasks 2 (Engaging Comment Classification) and 3 (Fact-Claiming Comment Classification). While acceptable results can be obtaine
d by using unigrams or linguistic features in combination with traditional machine learning models, we show that for both tasks transformer models trained on fine-tuned BERT embeddings yield best results.
This paper addresses the identification of toxic, engaging, and fact-claiming comments on social media. We used the dataset made available by the organizers of the GermEval2021 shared task containing over 3,000 manually annotated Facebook comments in
German. Considering the relatedness of the three tasks, we approached the problem using large pre-trained transformer models and multitask learning. Our results indicate that multitask learning achieves performance superior to the more common single task learning approach in all three tasks. We submit our best systems to GermEval-2021 under the team name WLV-RIT.
This paper provides an overview of the WANLP 2021 shared task on sarcasm and sentiment detection in Arabic. The shared task has two subtasks: sarcasm detection (subtask 1) and sentiment analysis (subtask 2). This shared task aims to promote and bring
attention to Arabic sarcasm detection, which is crucial to improve the performance in other tasks such as sentiment analysis. The dataset used in this shared task, namely ArSarcasm-v2, consists of 15,548 tweets labelled for sarcasm, sentiment and dialect. We received 27 and 22 submissions for subtasks 1 and 2 respectively. Most of the approaches relied on using and fine-tuning pre-trained language models such as AraBERT and MARBERT. The top achieved results for the sarcasm detection and sentiment analysis tasks were 0.6225 F1-score and 0.748 F1-PN respectively.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
