Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSelf-Alignment Pretraining for Biomedical Entity Representations
المحاذاة الذاتية محاكاة لتمثيل الكيانات الطبية الحيوية
399
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
على الرغم من النجاح الواسع النطاق للتعلم الإشراف على الذات من خلال نماذج لغة ملثم (MLM)، فإن التقاط علاقات الدلالية الدقيقة الدقيقة في المجال الطبي الحيوي يظل تحديا. هذا أمر بالغ الأهمية لمهام مستوى الكيان مثل الكيان الذي يربط حيث القدرة على نموذج العلاقات الكيانية (خاصة المرادف) محورية. لمعالجة هذا التحدي، نقترح Sapbert، وهو مخطط الاحتجاط بأنه يتماشى على مساحة التمثيل من الكيانات الطبية الحيوية. نقوم بتصميم إطار تعلم متري قابل للتطوير الذي يمكنه الاستفادة من UMLs، وهي مجموعة هائلة من الأنتولوج الطبي الطبيعي مع مفاهيم 4M +. على النقيض من ذلك مع أنظمة Hybrid التي تعتمد على الأنابيب السابقة، تقدم Sapbert حلا أنيقا نموذجيا لمشكلة ربط الكيان الطبي (MEL)، وتحقيق حالة جديدة من بين الفن (SOTA) على ستة MEL Benchmark مجموعات البيانات. في المجال العلمي، نحقق سوتا حتى بدون إشراف خاص بمهام المهام. مع تحسن كبير على مختلف MLMS المحدد مسبقا للمجال مثل BioBert و SciberTand و PubMedbert، يثبت نظامنا المحدد فعال وقوي.
Despite the widespread success of self-supervised learning via masked language models (MLM), accurately capturing fine-grained semantic relationships in the biomedical domain remains a challenge. This is of paramount importance for entity-level tasks such as entity linking where the ability to model entity relations (especially synonymy) is pivotal. To address this challenge, we propose SapBERT, a pretraining scheme that self-aligns the representation space of biomedical entities. We design a scalable metric learning framework that can leverage UMLS, a massive collection of biomedical ontologies with 4M+ concepts. In contrast with previous pipeline-based hybrid systems, SapBERT offers an elegant one-model-for-all solution to the problem of medical entity linking (MEL), achieving a new state-of-the-art (SOTA) on six MEL benchmarking datasets. In the scientific domain, we achieve SOTA even without task-specific supervision. With substantial improvement over various domain-specific pretrained MLMs such as BioBERT, SciBERTand and PubMedBERT, our pretraining scheme proves to be both effective and robust.
References used
https://aclanthology.org/
rate research
Read More
Introducing biomedical informatics (BMI) students to natural language processing (NLP) requires balancing technical depth with practical know-how to address application-focused needs. We developed a set of three activities introducing introductory BM
I students to information retrieval with NLP, covering document representation strategies and language models from TF-IDF to BERT. These activities provide students with hands-on experience targeted towards common use cases, and introduce fundamental components of NLP workflows for a wide variety of applications.
In this paper we present the FJWU's system submitted to the biomedical shared task at WMT21. We prepared state-of-the-art multilingual neural machine translation systems for three languages (i.e. German, Spanish and French) with English as target lan
guage. Our NMT systems based on Transformer architecture, were trained on combination of in-domain and out-domain parallel corpora developed using Information Retrieval (IR) and domain adaptation techniques.
Cross-lingual entity alignment (EA) aims to find the equivalent entities between crosslingual KGs (Knowledge Graphs), which is a crucial step for integrating KGs. Recently, many GNN-based EA methods are proposed and show decent performance improvemen
ts on several public datasets. However, existing GNN-based EA methods inevitably inherit poor interpretability and low efficiency from neural networks. Motivated by the isomorphic assumption of GNN-based methods, we successfully transform the cross-lingual EA problem into an assignment problem. Based on this re-definition, we propose a frustratingly Simple but Effective Unsupervised entity alignment method (SEU) without neural networks. Extensive experiments have been conducted to show that our proposed unsupervised approach even beats advanced supervised methods across all public datasets while having high efficiency, interpretability, and stability.
Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural language processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and ef
ficient framework to nominate candidates if entities are only identified by descriptions. However, they ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method. Our method achieves the state-of-the-art performance on ZESHEL and improves the quality of candidates on three standard Entity Linking datasets.
This paper describes the Fujitsu DMATH systems used for WMT 2021 News Translation and Biomedical Translation tasks. We focused on low-resource pairs, using a simple system. We conducted experiments on English-Hausa, Xhosa-Zulu and English-Basque, and
submitted the results for Xhosa→Zulu in the News Translation Task, and English→Basque in the Biomedical Translation Task, abstract and terminology translation subtasks. Our system combines BPE dropout, sub-subword features and back-translation with a Transformer (base) model, achieving good results on the evaluation sets.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
