Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTo Block or not to Block: Experiments with Machine Learning for News Comment Moderation
لحظر أو عدم حظره: تجارب مع التعلم الآلي للإعلام التعليق الأخبار
305
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
اليوم، تتضمن المؤسسات الإعلامية الأخبار بانتظام مع القراء من خلال تمكينهم من التعليق على المقالات الإخبارية.هذا يخلق الحاجة إلى التعليق الاعتدال وإزالة التعليقات غير المسموح بها - وهي مهمة تستغرق وقتا طويلا في كثير من الأحيان أداءها المشرفين البشري.في هذه الورقة، نقترب من مشكلة اعتدال تعليق الأخبار التلقائي كتصنيف للتعليقات في الفئات المحظورة وغير المحظورة.نبني مجموعة بيانات جديدة من تعليقات باللغة الإنجليزية المشروح، وتجربة نقل متصل بالتعليق، وتقييم العديد من نماذج التعلم الآلي على مجموعات بيانات الأخبار الكرواتية والإستونية.اسم الفريق: SuperAdmin؛التحدي: اكتشاف التعليقات المحظورة؛أدوات / نماذج: Brrosloen Bert، أرقى بيرت، 24sata تعليق DataSet، Ekspress تعليق DataSet.
Today, news media organizations regularly engage with readers by enabling them to comment on news articles. This creates the need for comment moderation and removal of disallowed comments -- a time-consuming task often performed by human moderators. In this paper we approach the problem of automatic news comment moderation as classification of comments into blocked and not blocked categories. We construct a novel dataset of annotated English comments, experiment with cross-lingual transfer of comment labels and evaluate several machine learning models on datasets of Croatian and Estonian news comments. Team name: SuperAdmin; Challenge: Detection of blocked comments; Tools/models: CroSloEn BERT, FinEst BERT, 24Sata comment dataset, Ekspress comment dataset.
References used
https://aclanthology.org/
rate research
Read More
Automatic metrics are commonly used as the exclusive tool for declaring the superiority of one machine translation system's quality over another. The community choice of automatic metric guides research directions and industrial developments by decid
ing which models are deemed better. Evaluating metrics correlations with sets of human judgements has been limited by the size of these sets. In this paper, we corroborate how reliable metrics are in contrast to human judgements on -- to the best of our knowledge -- the largest collection of judgements reported in the literature. Arguably, pairwise rankings of two systems are the most common evaluation tasks in research or deployment scenarios. Taking human judgement as a gold standard, we investigate which metrics have the highest accuracy in predicting translation quality rankings for such system pairs. Furthermore, we evaluate the performance of various metrics across different language pairs and domains. Lastly, we show that the sole use of BLEU impeded the development of improved models leading to bad deployment decisions. We release the collection of 2.3M sentence-level human judgements for 4380 systems for further analysis and replication of our work.
In recent years, time-critical processing or real-time processing and analytics of bid data have received a significant amount of attentions. There are many areas/domains where real-time processing of data and making timely decision can save thousand
s of human lives, minimizing the risks of human lives and resources, enhance the quality of human lives, enhance the chance of profitability, efficient resources management etc. This paper has presented such type of real-time big data analytic applications and a classification of those applications. In addition, it presents the time requirements of each type of these applications along with its significant benefits. Also, a general overview of big data to describe a background knowledge on this scope.
Moderation of reader comments is a significant problem for online news platforms. Here, we experiment with models for automatic moderation, using a dataset of comments from a popular Croatian newspaper. Our analysis shows that while comments that vio
late the moderation rules mostly share common linguistic and thematic features, their content varies across the different sections of the newspaper. We therefore make our models topic-aware, incorporating semantic features from a topic model into the classification decision. Our results show that topic information improves the performance of the model, increases its confidence in correct outputs, and helps us understand the model's outputs.
Recent work has demonstrated that pre-training in-domain language models can boost performance when adapting to a new domain. However, the costs associated with pre-training raise an important question: given a fixed budget, what steps should an NLP
practitioner take to maximize performance? In this paper, we study domain adaptation under budget constraints, and approach it as a customer choice problem between data annotation and pre-training. Specifically, we measure the annotation cost of three procedural text datasets and the pre-training cost of three in-domain language models. Then we evaluate the utility of different combinations of pre-training and data annotation under varying budget constraints to assess which combination strategy works best. We find that, for small budgets, spending all funds on annotation leads to the best performance; once the budget becomes large enough, a combination of data annotation and in-domain pre-training works more optimally. We therefore suggest that task-specific data annotation should be part of an economical strategy when adapting an NLP model to a new domain.
Recently, pre-trained language representation models such as BERT and RoBERTa have achieved significant results in a wide range of natural language processing (NLP) tasks, however, it requires extremely high computational cost. Curriculum Learning (C
L) is one of the potential solutions to alleviate this problem. CL is a training strategy where training samples are given to models in a meaningful order instead of random sampling. In this work, we propose a new CL method which gradually increases the block-size of input text for training the self-attention mechanism of BERT and its variants using the maximum available batch-size. Experiments in low-resource settings show that our approach outperforms the baseline in terms of convergence speed and final performance on downstream tasks.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
