Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userUni-FedRec: A Unified Privacy-Preserving News Recommendation Framework for Model Training and Online Serving
UNI-FEDREC: إطار توصية أخبار محافظة الخصوصية الموحدة للتدريب النموذجي والخدمة عبر الإنترنت
343
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يمكن أن تساعد تقنيات توصيات الأخبار المستخدمين على منصات الأخبار للحصول على معلومات الأخبار المفضلة لديهم. تعتمد معظم طرق توصيات الأخبار الحالية على بيانات سلوك المستخدم المخزنة مركزيا لتدريب نماذج وخدمة المستخدمين. ومع ذلك، فإن بيانات المستخدم عادة ما تكون حساسة خصوصية عالية، وتخزينها مركزيا في منصة الأخبار قد تثير مخاوف الخصوصية والمخاطر. في هذه الورقة، نقترح إطار توصية أخبار موحدة، والتي يمكن أن تستخدم بيانات المستخدم المخزنة محليا في عملاء المستخدمين لتدريب النماذج وخدمة المستخدمين بطريقة محافظة الخصوصية. بعد النموذج المستخدمة على نطاق واسع في أنظمة التوصية في العالم الحقيقي، يحتوي إطار عملنا على مرحلة للجيل الأخبار المرشح (I.E.، استدعاء) ومرحلة لترتيب الأخبار المرشح (أي، الترتيب). في مرحلة الاستدعاء، يتعلم كل عميل محليا تمثيلات فائدة متعددة من الأخبار النقر باهتمامات المستخدم النموذجية الشاملة. تم تحميل هذه التمثيلات إلى الخادم لاستدعاء أخبار المرشحين من تجمع أخبار كبير، والتي يتم توزيعها بشكل إضافي على عميل المستخدم في مرحلة الترتيب لعرض الأخبار المخصص. بالإضافة إلى ذلك، نقترح طريقة Decomposer-Decomposer-Decomposer مع ضوضاء الاضطرابات لتحسين حماية معلومات المستخدم الخاصة المشفرة في تمثيلات اهتمام المستخدم. علاوة على ذلك، فإننا نتدرب بشكل تعاوني في تذكر نماذج الترتيب والترتيب على البيانات اللامركزية في عدد كبير من عملاء المستخدمين بطريقة الحفاظ على الخصوصية. تبين التجارب في مجموعات بيانات الأخبار الحقيقية في العالم أن طريقتنا يمكن أن تفوق أساليب خط الأساس وتحمي خصوصية المستخدم بشكل فعال.
News recommendation techniques can help users on news platforms obtain their preferred news information. Most existing news recommendation methods rely on centrally stored user behavior data to train models and serve users. However, user data is usually highly privacy-sensitive, and centrally storing them in the news platform may raise privacy concerns and risks. In this paper, we propose a unified news recommendation framework, which can utilize user data locally stored in user clients to train models and serve users in a privacy-preserving way. Following a widely used paradigm in real-world recommender systems, our framework contains a stage for candidate news generation (i.e., recall) and a stage for candidate news ranking (i.e., ranking). At the recall stage, each client locally learns multiple interest representations from clicked news to comprehensively model user interests. These representations are uploaded to the server to recall candidate news from a large news pool, which are further distributed to the user client at the ranking stage for personalized news display. In addition, we propose an interest decomposer-aggregator method with perturbation noise to better protect private user information encoded in user interest representations. Besides, we collaboratively train both recall and ranking models on the data decentralized in a large number of user clients in a privacy-preserving way. Experiments on two real-world news datasets show that our method can outperform baseline methods and effectively protect user privacy.
References used
https://aclanthology.org/
rate research
Read More
News recommendation is critical for personalized news access. Most existing news recommendation methods rely on centralized storage of users' historical news click behavior data, which may lead to privacy concerns and hazards. Federated Learning is a
privacy-preserving framework for multiple clients to collaboratively train models without sharing their private data. However, the computation and communication cost of directly learning many existing news recommendation models in a federated way are unacceptable for user clients. In this paper, we propose an efficient federated learning framework for privacy-preserving news recommendation. Instead of training and communicating the whole model, we decompose the news recommendation model into a large news model maintained in the server and a light-weight user model shared on both server and clients, where news representations and user model are communicated between server and clients. More specifically, the clients request the user model and news representations from the server, and send their locally computed gradients to the server for aggregation. The server updates its global user model with the aggregated gradients, and further updates its news model to infer updated news representations. Since the local gradients may contain private information, we propose a secure aggregation method to aggregate gradients in a privacy-preserving way. Experiments on two real-world datasets show that our method can reduce the computation and communication cost on clients while keep promising model performance.
Latent Dirichlet allocation (LDA), a widely used topic model, is often employed as a fundamental tool for text analysis in various applications. However, the training process of the LDA model typically requires massive text corpus data. On one hand,
such massive data may expose private information in the training data, thereby incurring significant privacy concerns. On the other hand, the efficiency of the LDA model training may be impacted, since LDA training often needs to handle these massive text corpus data. To address the privacy issues in LDA model training, some recent works have combined LDA training algorithms that are based on collapsed Gibbs sampling (CGS) with differential privacy. Nevertheless, these works usually have a high accumulative privacy budget due to vast iterations in CGS. Moreover, these works always have low efficiency due to handling massive text corpus data. To improve the privacy guarantee and efficiency, we combine a subsampling method with CGS and propose a novel LDA training algorithm with differential privacy, SUB-LDA. We find that subsampling in CGS naturally improves efficiency while amplifying privacy. We propose a novel metric, the efficiency--privacy function, to evaluate improvements of the privacy guarantee and efficiency. Based on a conventional subsampling method, we propose an adaptive subsampling method to improve the model's utility produced by SUB-LDA when the subsampling ratio is small. We provide a comprehensive analysis of SUB-LDA, and the experiment results validate its efficiency and privacy guarantee improvements.
Modern deep learning models for natural language processing rely heavily on large amounts of annotated texts. However, obtaining such texts may be difficult when they contain personal or confidential information, for example, in health or legal domai
ns. In this work, we propose a method of de-identifying free-form text documents by carefully redacting sensitive data in them. We show that our method preserves data utility for text classification, sequence labeling and question answering tasks.
In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts. The unified view helps us to better understand the ex
isting methods for learning cross-lingual representations. More importantly, inspired by the framework, we propose a new pre-training task based on contrastive learning. Specifically, we regard a bilingual sentence pair as two views of the same meaning and encourage their encoded representations to be more similar than the negative examples. By leveraging both monolingual and parallel corpora, we jointly train the pretext tasks to improve the cross-lingual transferability of pre-trained models. Experimental results on several benchmarks show that our approach achieves considerably better performance. The code and pre-trained models are available at https://aka.ms/infoxlm.
For programmers, learning the usage of APIs (Application Programming Interfaces) of a software library is important yet difficult. API recommendation tools can help developers use APIs by recommending which APIs to be used next given the APIs that ha
ve been written. Traditionally, language models such as N-gram are applied to API recommendation. However, because the software libraries keep changing and new libraries keep emerging, new APIs are common. These new APIs can be seen as OOV (out of vocabulary) words and cannot be handled well by existing API recommendation approaches due to the lack of training data. In this paper, we propose APIRecX, the first cross-library API recommendation approach, which uses BPE to split each API call in each API sequence and pre-trains a GPT based language model. It then recommends APIs by fine-tuning the pre-trained model. APIRecX can migrate the knowledge of existing libraries to a new library, and can recommend APIs that are previously regarded as OOV. We evaluate APIRecX on six libraries and the results confirm its effectiveness by comparing with two typical API recommendation approaches.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
