Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEfficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation
كفاءة-فيدريك: إطار التعلم الفيدرالي الفعال لتوصية الأخبار المحفوظة للخصوصية
342
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
توصية الأخبار أمر بالغ الأهمية للوصول إلى الأخبار المخصصة. تعتمد أساليب توصيات الأخبار الموجودة على معظم طرق التخزين المركزي للأخبار التاريخية للمستخدمين النقر فوق بيانات السلوك، والتي قد تؤدي إلى مخاوف الخصوصية والمخاطر. يعد التعلم الفيدرالي إطارا للحفاظ على الخصوصية لعملاء متعددين نماذج قطار تعاوني دون مشاركة بياناتهم الخاصة. ومع ذلك، فإن حساب التكاليف والاتصال في تعلم العديد من نماذج توصية الأخبار الموجودة بطريقة غير مقبولة غير مقبولة لعملاء المستخدمين. في هذه الورقة، نقترح إطارا تعليميا فائضا فعالا لتوصية الأخبار التي تحافظ على الخصوصية. بدلا من تدريب وتوصيل النموذج بأكمله، نقوم بتحلل نموذج توصية الأخبار إلى نموذج أخبار كبير يحتفظ به في الخادم ونموذج مستخدم إضاءة الوزن مشتركا على كل من الخادم والعملاء، حيث يتم توصيل تمثيل الأخبار ونموذج المستخدم بين الخادم والعملاء وبعد وبشكل أكثر تحديدا، يطلب العملاء طراز المستخدم والتمثيلات الأخبار من الخادم، وإرسال تدرجاتهم المحسوبة محليا إلى الخادم للتجميع. يقوم الخادم بتحديث نموذج المستخدم العالمي الخاص به مع التدرجات المجمعة، ويقوم كذلك بتحديث نموذج الأخبار الخاص به لاستنتاج تمثيلات أخبار محدثة. نظرا لأن التدرجات المحلية قد تحتوي على معلومات خاصة، فإننا نقترح طريقة تجميع آمنة للتدرجات الإجمالية في طريقة الحفاظ على الخصوصية. تظهر التجارب في مجموعات بيانات عالمية حقيقية أن طريقتنا يمكن أن تقلل من حساب حساب الاتصالات والاتصال على العملاء مع الحفاظ على أداء نموذج واعد.
News recommendation is critical for personalized news access. Most existing news recommendation methods rely on centralized storage of users' historical news click behavior data, which may lead to privacy concerns and hazards. Federated Learning is a privacy-preserving framework for multiple clients to collaboratively train models without sharing their private data. However, the computation and communication cost of directly learning many existing news recommendation models in a federated way are unacceptable for user clients. In this paper, we propose an efficient federated learning framework for privacy-preserving news recommendation. Instead of training and communicating the whole model, we decompose the news recommendation model into a large news model maintained in the server and a light-weight user model shared on both server and clients, where news representations and user model are communicated between server and clients. More specifically, the clients request the user model and news representations from the server, and send their locally computed gradients to the server for aggregation. The server updates its global user model with the aggregated gradients, and further updates its news model to infer updated news representations. Since the local gradients may contain private information, we propose a secure aggregation method to aggregate gradients in a privacy-preserving way. Experiments on two real-world datasets show that our method can reduce the computation and communication cost on clients while keep promising model performance.
References used
https://aclanthology.org/
rate research
Read More
News recommendation techniques can help users on news platforms obtain their preferred news information. Most existing news recommendation methods rely on centrally stored user behavior data to train models and serve users. However, user data is usua
lly highly privacy-sensitive, and centrally storing them in the news platform may raise privacy concerns and risks. In this paper, we propose a unified news recommendation framework, which can utilize user data locally stored in user clients to train models and serve users in a privacy-preserving way. Following a widely used paradigm in real-world recommender systems, our framework contains a stage for candidate news generation (i.e., recall) and a stage for candidate news ranking (i.e., ranking). At the recall stage, each client locally learns multiple interest representations from clicked news to comprehensively model user interests. These representations are uploaded to the server to recall candidate news from a large news pool, which are further distributed to the user client at the ranking stage for personalized news display. In addition, we propose an interest decomposer-aggregator method with perturbation noise to better protect private user information encoded in user interest representations. Besides, we collaboratively train both recall and ranking models on the data decentralized in a large number of user clients in a privacy-preserving way. Experiments on two real-world news datasets show that our method can outperform baseline methods and effectively protect user privacy.
In this work, we consider the problem of designing secure and efficient federated learning (FL) frameworks for NLP. Existing solutions under this literature either consider a trusted aggregator or require heavy-weight cryptographic primitives, which
makes the performance significantly degraded. Moreover, many existing secure FL designs work only under the restrictive assumption that none of the clients can be dropped out from the training protocol. To tackle these problems, we propose SEFL, a secure and efficient federated learning framework that (1) eliminates the need for the trusted entities; (2) achieves similar and even better model accuracy compared with existing FL designs; (3) is resilient to client dropouts.
We curated WikiPII, an automatically labeled dataset composed of Wikipedia biography pages, annotated for personal information extraction. Although automatic annotation can lead to a high degree of label noise, it is an inexpensive process and can ge
nerate large volumes of annotated documents. We trained a BERT-based NER model with WikiPII and showed that with an adequately large training dataset, the model can significantly decrease the cost of manual information extraction, despite the high level of label noise. In a similar approach, organizations can leverage text mining techniques to create customized annotated datasets from their historical data without sharing the raw data for human annotation. Also, we explore collaborative training of NER models through federated learning when the annotation is noisy. Our results suggest that depending on the level of trust to the ML operator and the volume of the available data, distributed training can be an effective way of training a personal information identifier in a privacy-preserved manner. Research material is available at https://github.com/ratmcu/wikipiifed.
Due to complex cognitive and inferential efforts involved in the manual generation of one caption per image/video input, the human annotation resources are very limited for captioning tasks. We define language resource efficient as reaching the same
performance with fewer annotated captions per input. We first study the performance degradation of caption models in different language resource settings. Our analysis of caption models with SC loss shows that the performance degradation is caused by the increasingly noisy estimation of reward and baseline with fewer language resources. To mitigate this issue, we propose to reduce the variance of noise in the baseline by generalizing the single pairwise comparison in SC loss and using multiple generalized pairwise comparisons. The generalized pairwise comparison (GPC) measures the difference between the evaluation scores of two captions with respect to an input. Empirically, we show that the model trained with the proposed GPC loss is efficient on language resource and achieves similar performance with the state-of-the-art models on MSCOCO by using only half of the language resources. Furthermore, our model significantly outperforms the state-of-the-art models on a video caption dataset that has only one labeled caption per input in the training set.
The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides t
o effectively pinpoint salient information from the source. We further conduct a systematic study of existing efficient self-attentions. Combined with Hepos, we are able to process ten times more tokens than existing models that use full attentions. For evaluation, we present a new dataset, GovReport, with significantly longer documents and summaries. Results show that our models produce significantly higher ROUGE scores than competitive comparisons, including new state-of-the-art results on PubMed. Human evaluation also shows that our models generate more informative summaries with fewer unfaithful errors.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
