Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTraining Language Models under Resource Constraints for Adversarial Advertisement Detection
نماذج لغة التدريب تحت قيود الموارد للكشف عن المعددي
415
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
الإعلان عن التجارة الإلكترونية ومواقع وسائل التواصل الاجتماعي تقدم انطباعات إعلانية على نطاق الويب بقيمة قيادة يومية لكل من المتسوقين والمعلنين.يتطلب هذا المقياس طرقا برنامجية للكشف عن محتوى غير مناسب في الإعلانات لحماية تجربة العملاء والثقة.يركز هذه الورق على تقنيات تدريب نماذج تصنيف النص بموجب قيود الموارد، بنيت كجزء من الحلول الآلية للاعتدال محتوى الإعلان.نظير على مدى ضعف الإشراف، يمكن تطبيق تعلم المناهج الدراسية والتدريب متعدد اللغات بفعالية على Tune-Tune Bert ومتغيراته لمهام تصنيف النص بالاقتران مع مختلف استراتيجيات تكبير البيانات.تبين تجاربنا الواسعة لغات متعددة أن هذه التقنيات تكتشف فئات الإعلانات العدائية مع مكاسب كبيرة في الدقة في عتبة استدعاء عالية فوق الأساس.
Advertising on e-commerce and social media sites deliver ad impressions at web scale on a daily basis driving value to both shoppers and advertisers. This scale necessitates programmatic ways of detecting unsuitable content in ads to safeguard customer experience and trust. This paper focusses on techniques for training text classification models under resource constraints, built as part of automated solutions for advertising content moderation. We show how weak supervision, curriculum learning and multi-lingual training can be applied effectively to fine-tune BERT and its variants for text classification tasks in conjunction with different data augmentation strategies. Our extensive experiments on multiple languages show that these techniques detect adversarial ad categories with a substantial gain in precision at high recall threshold over the baseline.
References used
https://aclanthology.org/
rate research
Read More
Cross-target generalization constitutes an important issue for news Stance Detection (SD). In this short paper, we investigate adversarial cross-genre SD, where knowledge from annotated user-generated data is leveraged to improve news SD on targets u
nseen during training. We implement a BERT-based adversarial network and show experimental performance improvements over a set of strong baselines. Given the abundance of user-generated data, which are considerably less expensive to retrieve and annotate than news articles, this constitutes a promising research direction.
Large language models have become increasingly difficult to train because of the growing computation time and cost. In this work, we present SRU++, a highly-efficient architecture that combines fast recurrence and attention for sequence modeling. SRU
++ exhibits strong modeling capacity and training efficiency. On standard language modeling tasks such as Enwik8, Wiki-103 and Billion Word datasets, our model obtains better bits-per-character and perplexity while using 3x-10x less training cost compared to top-performing Transformer models. For instance, our model achieves a state-of-the-art result on the Enwik8 dataset using 1.6 days of training on an 8-GPU machine. We further demonstrate that SRU++ requires minimal attention for near state-of-the-art performance. Our results suggest jointly leveraging fast recurrence with little attention as a promising direction for accelerating model training and inference.
Detecting stance on Twitter is especially challenging because of the short length of each tweet, the continuous coinage of new terminology and hashtags, and the deviation of sentence structure from standard prose. Fine-tuned language models using lar
ge-scale in-domain data have been shown to be the new state-of-the-art for many NLP tasks, including stance detection. In this paper, we propose a novel BERT-based fine-tuning method that enhances the masked language model for stance detection. Instead of random token masking, we propose using a weighted log-odds-ratio to identify words with high stance distinguishability and then model an attention mechanism that focuses on these words. We show that our proposed approach outperforms the state of the art for stance detection on Twitter data about the 2020 US Presidential election.
The problem of detecting psychological stress in online posts, and more broadly, of detecting people in distress or in need of help, is a sensitive application for which the ability to interpret models is vital. Here, we present work exploring the us
e of a semantically related task, emotion detection, for equally competent but more explainable and human-like psychological stress detection as compared to a black-box model. In particular, we explore the use of multi-task learning as well as emotion-based language model fine-tuning. With our emotion-infused models, we see comparable results to state-of-the-art BERT. Our analysis of the words used for prediction show that our emotion-infused models mirror psychological components of stress.
We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to improve adversarial training for natural language understanding. The key idea is to introspect current mistakes and prioritize adversarial training steps to where the
model errs the most. Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released upon acceptance of the paper.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
