في تطوير نظام لإجواب الأسئلة عبر الإنترنت للنطاقات الطبية، تلعب نماذج الاستدلال باللغة الطبيعية (NLI) دورا رئيسيا في مطابقة الأسئلة والكشف عن النية.ومع ذلك، ما هي النماذج هي الأفضل لمجموعات البيانات لدينا؟اختيار أو ضبط نموذج يدويا هو تستغرق وقتا طويلا.وبالتالي، نقوم بتجربة تحسين هياكل النموذج تلقائيا على المهمة في متناول اليد عبر بحث العمارة العصبية (NAS).أولا، نقوم بصياغة مساحة بحث عن الهندسة المعمارية الجديدة تعتمد على أدب NAS السابق، ودعم النمذجة الانتباه عبر الجملة (Cross-Attn).ثانيا، نقترح تعديل طريقة ENAS لتسريع وتحقيق الاستقرار في نتائج البحث.نقوم بإجراء تجارب واسعة النطاق على مهامنا NLI الطبية.تظهر النتائج أن نظامنا يمكن أن يتفوق بسهولة على نماذج الأساس الكلاسيكية.نقارن أساليب NAS المختلفة وإظهار نهجنا يوفر أفضل النتائج.
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection. However, which models are best for our datasets? Manually selecting or tuning a model is time-consuming. Thus we experiment with automatically optimizing the model architectures on the task at hand via neural architecture search (NAS). First, we formulate a novel architecture search space based on the previous NAS literature, supporting cross-sentence attention (cross-attn) modeling. Second, we propose to modify the ENAS method to accelerate and stabilize the search results. We conduct extensive experiments on our two medical NLI tasks. Results show that our system can easily outperform the classical baseline models. We compare different NAS methods and demonstrate our approach provides the best results.
References used
https://aclanthology.org/
Healthcare is becoming a more and more important research topic recently. With the growing data in the healthcare domain, it offers a great opportunity for deep learning to improve the quality of service and reduce costs. However, the complexity of e
Query Rewriting (QR) is proposed to solve the problem of the word mismatch between queries and documents in Web search. Existing approaches usually model QR with an end-to-end sequence-to-sequence (seq2seq) model. The state-of-the-art Transformer-bas
In this paper, we propose a definition and taxonomy of various types of non-standard textual content -- generally referred to as noise'' -- in Natural Language Processing (NLP). While data pre-processing is undoubtedly important in NLP, especially wh
Existing pre-trained language models (PLMs) have demonstrated the effectiveness of self-supervised learning for a broad range of natural language processing (NLP) tasks. However, most of them are not explicitly aware of domain-specific knowledge, whi
Lack of training data presents a grand challenge to scaling out spoken language understanding (SLU) to low-resource languages. Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languag