Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userGeo-BERT Pre-training Model for Query Rewriting in POI Search
Geo-Bert Pre-Training Model لإعادة كتابة الاستعلام في بحث POI
283
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يقترح إعادة كتابة الاستعلام (QR) حل مشكلة عدم تطابق الكلمة بين الاستفسارات والمستندات في البحث على الويب. الأساليب الحالية عادة ما نموذج QR مع نموذج تسلسل نهاية إلى نهاية (SEQ2SEQ). يمكن أن تتعلم النماذج القائمة على المحولات الحديثة بفعالية دلالات نصية من سجلات جلسة المستخدم، لكنها غالبا ما تتجاهل معلومات الموقع الجغرافي للمستخدمين الحيوية لتحقيق البحث عن نقطة الفائدة (POI) على خدمات الخريطة. في هذه الورقة، اقترحنا نموذجا ما قبل التدريب، يسمى GEO-BERT، لدمج الدلالات والمعلومات الجغرافية في التمثيلات المدربة مسبقا للويس. أولا، نحاكي توزيع POI في العالم الحقيقي كشركة رسم بياني، حيث تمثل العقد Pois ومتعدد الحبيبات الجغرافية. ثم نستخدم أساليب تعلم التمثيل الرسم البياني للحصول على تمثيلات جغرافية. أخيرا، نحن ندرب نموذجا ما قبل التدريب يشبه بيرت مع تضيير الرسوم البيانية النصية والنصية للحصول على تمثيل متكامل لكل من المعلومات الجغرافية والدلية، وتطبيقه في البحث عن QR of POI. يحقق النموذج المقترح دقة ممتازة على مجموعة واسعة من مجموعات بيانات خريطة العالم الواقعية.
Query Rewriting (QR) is proposed to solve the problem of the word mismatch between queries and documents in Web search. Existing approaches usually model QR with an end-to-end sequence-to-sequence (seq2seq) model. The state-of-the-art Transformer-based models can effectively learn textual semantics from user session logs, but they often ignore users' geographic location information that is crucial for the Point-of-Interest (POI) search of map services. In this paper, we proposed a pre-training model, called Geo-BERT, to integrate semantics and geographic information in the pre-trained representations of POIs. Firstly, we simulate POI distribution in the real world as a graph, in which nodes represent POIs and multiple geographic granularities. Then we use graph representation learning methods to get geographic representations. Finally, we train a BERT-like pre-training model with text and POIs' graph embeddings to get an integrated representation of both geographic and semantic information, and apply it in the QR of POI search. The proposed model achieves excellent accuracy on a wide range of real-world datasets of map services.
References used
https://aclanthology.org/
rate research
Read More
Query rewrite (QR) is an emerging component in conversational AI systems, reducing user defect. User defect is caused by various reasons, such as errors in the spoken dialogue system, users' slips of the tongue or their abridged language. Many of the
user defects stem from personalized factors, such as user's speech pattern, dialect, or preferences. In this work, we propose a personalized search-based QR framework, which focuses on automatic reduction of user defect. We build a personalized index for each user, which encompasses diverse affinity layers to reflect personal preferences for each user in the conversational AI. Our personalized QR system contains retrieval and ranking layers. Supported by user feedback based learning, training our models does not require hand-annotated data. Experiments on personalized test set showed that our personalized QR system is able to correct systematic and user errors by utilizing phonetic and semantic inputs.
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection. However, which models are best for our datasets? Manually selecting o
r tuning a model is time-consuming. Thus we experiment with automatically optimizing the model architectures on the task at hand via neural architecture search (NAS). First, we formulate a novel architecture search space based on the previous NAS literature, supporting cross-sentence attention (cross-attn) modeling. Second, we propose to modify the ENAS method to accelerate and stabilize the search results. We conduct extensive experiments on our two medical NLI tasks. Results show that our system can easily outperform the classical baseline models. We compare different NAS methods and demonstrate our approach provides the best results.
This paper describes a system proposed for the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (EUD). We propose a Graph Rewriting based system for computing Enhanced Universal Dependencies, given the Basic Universal Dependencies (UD).
Large-scale pretrained transformer models have demonstrated state-of-the-art (SOTA) performance in a variety of NLP tasks. Nowadays, numerous pretrained models are available in different model flavors and different languages, and can be easily adapte
d to one's downstream task. However, only a limited number of models are available for dialogue tasks, and in particular, goal-oriented dialogue tasks. In addition, the available pretrained models are trained on general domain language, creating a mismatch between the pretraining language and the downstream domain launguage. In this contribution, we present CS-BERT, a BERT model pretrained on millions of dialogues in the customer service domain. We evaluate CS-BERT on several downstream customer service dialogue tasks, and demonstrate that our in-domain pretraining is advantageous compared to other pretrained models in both zero-shot experiments as well as in finetuning experiments, especially in a low-resource data setting.
This paper describes a compact and effective model for low-latency passage retrieval in conversational search based on learned dense representations. Prior to our work, the state-of-the-art approach uses a multi-stage pipeline comprising conversation
al query reformulation and information retrieval modules. Despite its effectiveness, such a pipeline often includes multiple neural models that require long inference times. In addition, independently optimizing each module ignores dependencies among them. To address these shortcomings, we propose to integrate conversational query reformulation directly into a dense retrieval model. To aid in this goal, we create a dataset with pseudo-relevance labels for conversational search to overcome the lack of training data and to explore different training strategies. We demonstrate that our model effectively rewrites conversational queries as dense representations in conversational search and open-domain question answering datasets. Finally, after observing that our model learns to adjust the L2 norm of query token embeddings, we leverage this property for hybrid retrieval and to support error analysis.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
