Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userMemes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset
الميمات في البرية: تقييم تعميم بيانات تحدي الميمات البغيضة
540
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تشكل الميمات البغيضة تحديا فريدا لأنظمة تعلم الآلات الحالية لأن رسالتهم مشتقة من كل من الطرائق النصية والمرئية.لهذا الغرض، أصدر Facebook تحدي الميمات البغيض، مجموعة بيانات من الميمات ذات التسميات التوضيحية النصية المستخلصة مسبقا، لكن من غير الواضح ما إذا كانت هذه الأمثلة الاصطناعية تعزز إلى الميمات في البرية ".في هذه الورقة، نقوم بجمع الميمات البغيضة وغير البغيضة من Pinterest لتقييم الأداء الخارجي على النماذج المدربة مسبقا على مجموعة بيانات Facebook.نجد أن الميمات في البرية "تختلف في جوانبين رئيسيين: 1) يجب استخراج التسميات التوضيحية عبر OCR، ضجيج حقن وتقليل الأداء من النماذج متعددة الوسائط، و 2) الميمات أكثر تنوعا من الميمات التقليدية، بما في ذلك لقطات من المحادثات أو النصفي خلفية عادية.هذه الورقة هكذا بمثابة التحقق من الواقع للمعيار الحالي للكشف عن الكراهية ومستقليها على الكشف عن الكراهية في العالم الحقيقي.
Hateful memes pose a unique challenge for current machine learning systems because their message is derived from both text- and visual-modalities. To this effect, Facebook released the Hateful Memes Challenge, a dataset of memes with pre-extracted text captions, but it is unclear whether these synthetic examples generalize to memes in the wild'. In this paper, we collect hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset. We find that memes in the wild' differ in two key aspects: 1) Captions must be extracted via OCR, injecting noise and diminishing performance of multimodal models, and 2) Memes are more diverse than traditional memes', including screenshots of conversations or text on a plain background. This paper thus serves as a reality-check for the current benchmark of hateful meme detection and its applicability for detecting real world hate.
References used
https://aclanthology.org/
rate research
Read More
We present the results and main findings of the shared task at WOAH 5 on hateful memes detection. The task include two subtasks relating to distinct challenges in the fine-grained detection of hateful memes: (1) the protected category attacked by the
meme and (2) the attack type. 3 teams submitted system description papers. This shared task builds on the hateful memes detection task created by Facebook AI Research in 2020.
The Shared Task on Hateful Memes is a challenge that aims at the detection of hateful content in memes by inviting the implementation of systems that understand memes, potentially by combining image and textual information. The challenge consists of
three detection tasks: hate, protected category and attack type. The first is a binary classification task, while the other two are multi-label classification tasks. Our participation included a text-based BERT baseline (TxtBERT), the same but adding information from the image (ImgBERT), and neural retrieval approaches. We also experimented with retrieval augmented classification models. We found that an ensemble of TxtBERT and ImgBERT achieves the best performance in terms of ROC AUC score in two out of the three tasks on our development set.
Memes are the combinations of text and images that are often humorous in nature. But, that may not always be the case, and certain combinations of texts and images may depict hate, referred to as hateful memes. This work presents a multimodal pipelin
e that takes both visual and textual features from memes into account to (1) identify the protected category (e.g. race, sex etc.) that has been attacked; and (2) detect the type of attack (e.g. contempt, slurs etc.). Our pipeline uses state-of-the-art pre-trained visual and textual representations, followed by a simple logistic regression classifier. We employ our pipeline on the Hateful Memes Challenge dataset with additional newly created fine-grained labels for protected category and type of attack. Our best model achieves an AUROC of 0.96 for identifying the protected category, and 0.97 for detecting the type of attack. We release our code at https://github.com/harisbinzia/HatefulMemes
This paper describes our submission (winning solution for Task A) to the Shared Task on Hateful Meme Detection at WOAH 2021. We build our system on top of a state-of-the-art system for binary hateful meme classification that already uses image tags s
uch as race, gender, and web entities. We add further metadata such as emotions and experiment with data augmentation techniques, as hateful instances are underrepresented in the data set.
Internet memes have become powerful means to transmit political, psychological, and socio-cultural ideas. Although memes are typically humorous, recent days have witnessed an escalation of harmful memes used for trolling, cyberbullying, and abuse. De
tecting such memes is challenging as they can be highly satirical and cryptic. Moreover, while previous work has focused on specific aspects of memes such as hate speech and propaganda, there has been little work on harm in general. Here, we aim to bridge this gap. In particular, we focus on two tasks: (i)detecting harmful memes, and (ii) identifying the social entities they target. We further extend the recently released HarMeme dataset, which covered COVID-19, with additional memes and a new topic: US politics. To solve these tasks, we propose MOMENTA (MultimOdal framework for detecting harmful MemEs aNd Their tArgets), a novel multimodal deep neural network that uses global and local perspectives to detect harmful memes. MOMENTA systematically analyzes the local and the global perspective of the input meme (in both modalities) and relates it to the background context. MOMENTA is interpretable and generalizable, and our experiments show that it outperforms several strong rivaling approaches.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
