Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDetecting Cross-Geographic Biases in Toxicity Modeling on Social Media
الكشف عن التحيزات عبر الجغرافية في نمذجة سمية على وسائل التواصل الاجتماعي
406
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعتمد منصات وسائل التواصل الاجتماعي عبر الإنترنت على نحو متزايد على تقنيات معالجة اللغة الطبيعية (NLP) للكشف عن محتوى مسيء على نطاق واسع من أجل تخفيف الأضرار التي يسببها لمستخدميها. ومع ذلك، فإن هذه التقنيات تعاني من مختلف تحيزات أخذ العينات والجمعية الموجودة في البيانات التدريبية، والتي غالبا ما تؤدي إلى أداء الفرعية على المحتوى ذي الصلة بالمجموعات المهمشة، مما يحتمل أن يؤدي إلى أضرار غير متناسبة تجاههم. ركزت دراسات حول مثل هذه التحيزات حتى الآن على حفنة فقط من محاور التباينات والمجموعات الفرعية التي تحتوي على التعليقات التوضيحية / المعجم المتاحة. وبالتالي، يتم تجاهل التحيزات المتعلقة بالسياقات غير الغربية إلى حد كبير في الأدبيات. في هذه الورقة، نقدم طريقة خاضعة للإشراف ضعيفا للكشف عن التحيزات المعجمية بقوة في السياقات الجغرافية الثقافية الأوسع نطاقا. من خلال دراسة الحالة حول نموذج كشف للسمية المتوفرة للجمهور، نوضح أن طريقتنا تحدد المجموعات البارزة من الأخطاء المتبادلة الجغرافية، وفي متابعة، توضح أن هذه التجمعات تعكس الأحكام الإنسانية من اللغة الهجومية واللغة الفعلية في تلك السياقات الجغرافية. نحن أيضا إجراء تحليل نموذج تدرب على مجموعة بيانات مع ملصقات الحقيقة الأرضية لفهم هذه التحيزات بشكل أفضل، وتقديم تجارب التخفيف الأولي.
Online social media platforms increasingly rely on Natural Language Processing (NLP) techniques to detect abusive content at scale in order to mitigate the harms it causes to their users. However, these techniques suffer from various sampling and association biases present in training data, often resulting in sub-par performance on content relevant to marginalized groups, potentially furthering disproportionate harms towards them. Studies on such biases so far have focused on only a handful of axes of disparities and subgroups that have annotations/lexicons available. Consequently, biases concerning non-Western contexts are largely ignored in the literature. In this paper, we introduce a weakly supervised method to robustly detect lexical biases in broader geo-cultural contexts. Through a case study on a publicly available toxicity detection model, we demonstrate that our method identifies salient groups of cross-geographic errors, and, in a follow up, demonstrate that these groupings reflect human judgments of offensive and inoffensive language in those geographic contexts. We also conduct analysis of a model trained on a dataset with ground truth labels to better understand these biases, and present preliminary mitigation experiments.
References used
https://aclanthology.org/
rate research
Read More
The framing of political issues can influence policy and public opinion. Even though the public plays a key role in creating and spreading frames, little is known about how ordinary people on social media frame political issues. By creating a new dat
aset of immigration-related tweets labeled for multiple framing typologies from political communication theory, we develop supervised models to detect frames. We demonstrate how users' ideology and region impact framing choices, and how a message's framing influences audience responses. We find that the more commonly-used issue-generic frames obscure important ideological and regional patterns that are only revealed by immigration-specific frames. Furthermore, frames oriented towards human interests, culture, and politics are associated with higher user engagement. This large-scale analysis of a complex social and linguistic phenomenon contributes to both NLP and social science research.
The speech act of complaining is used by humans to communicate a negative mismatch between reality and expectations as a reaction to an unfavorable situation. Linguistic theory of pragmatics categorizes complaints into various severity levels based o
n the face-threat that the complainer is willing to undertake. This is particularly useful for understanding the intent of complainers and how humans develop suitable apology strategies. In this paper, we study the severity level of complaints for the first time in computational linguistics. To facilitate this, we enrich a publicly available data set of complaints with four severity categories and train different transformer-based networks combined with linguistic information achieving 55.7 macro F1. We also jointly model binary complaint classification and complaint severity in a multi-task setting achieving new state-of-the-art results on binary complaint detection reaching up to 88.2 macro F1. Finally, we present a qualitative analysis of the behavior of our models in predicting complaint severity levels.
We present machine learning classifiers to automatically identify COVID-19 misinformation on social media in three languages: English, Bulgarian, and Arabic. We compared 4 multitask learning models for this task and found that a model trained with En
glish BERT achieves the best results for English, and multilingual BERT achieves the best results for Bulgarian and Arabic. We experimented with zero shot, few shot, and target-only conditions to evaluate the impact of target-language training data on classifier performance, and to understand the capabilities of different models to generalize across languages in detecting misinformation online. This work was performed as a submission to the shared task, NLP4IF 2021: Fighting the COVID-19 Infodemic. Our best models achieved the second best evaluation test results for Bulgarian and Arabic among all the participating teams and obtained competitive scores for English.
Cross-language authorship attribution is the challenging task of classifying documents by bilingual authors where the training documents are written in a different language than the evaluation documents. Traditional solutions rely on either translati
on to enable the use of single-language features, or language-independent feature extraction methods. More recently, transformer-based language models like BERT can also be pre-trained on multiple languages, making them intuitive candidates for cross-language classifiers which have not been used for this task yet. We perform extensive experiments to benchmark the performance of three different approaches to a smallscale cross-language authorship attribution experiment: (1) using language-independent features with traditional classification models, (2) using multilingual pre-trained language models, and (3) using machine translation to allow single-language classification. For the language-independent features, we utilize universal syntactic features like part-of-speech tags and dependency graphs, and multilingual BERT as a pre-trained language model. We use a small-scale social media comments dataset, closely reflecting practical scenarios. We show that applying machine translation drastically increases the performance of almost all approaches, and that the syntactic features in combination with the translation step achieve the best overall classification performance. In particular, we demonstrate that pre-trained language models are outperformed by traditional models in small scale authorship attribution problems for every language combination analyzed in this paper.
Nowadays, there are a lot of advertisements hiding as normal posts or experience sharing in social media. There is little research of advertorial detection on Mandarin Chinese texts. This paper thus aimed to focus on hidden advertorial detection of o
nline posts in Taiwan Mandarin Chinese. We inspected seven contextual features based on linguistic theories in discourse level. These features can be further grouped into three schemas under the general advertorial writing structure. We further implemented these features to train a multi-task BERT model to detect advertorials. The results suggested that specific linguistic features would help extract advertorials.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
