Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userLeveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations
الاستفادة من النماذج المحددة للتلخيص التلقائي لمحادثات الطبيب المريض
281
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يعرض نماذج Resunding Runing Running لتلخيص محادثة محادثة الطبيب تلقائيا العديد من التحديات: بيانات تدريب محدودة، ونقل مجال كبير، والنصوص الطويلة والصعارية، والتقلبات الموجزة عالية الهدف. في هذه الورقة، نستكشف جدوى استخدام نماذج المحولات مسبقا لتلخيص محادثات الطبيب المريض تلقائيا مباشرة من النصوص. نظهر أنه يمكن إنشاء ملخصات بطلاقة وكافية بيانات تدريبية محدودة من قبل BARTING BART على مجموعة بيانات شيدة خصيصا. تتجاوز النماذج الناتجة بشكل كبير أداء Annotator البشري المتوسط ونوعية العمل المنشور السابق للمهمة. نقيم طرق متعددة للتعامل مع المحادثات الطويلة، ومقارنتها إلى خط الأساس الواضح لاقتطاع المحادثة لتناسب حد الطول المحدد مسبقا. نقدم نهجا متعدد المراحل يتناول المهمة من خلال تعلم اثنين من النماذج الدقيقة: واحد لتلخيص قطع المحادثة في ملخصات جزئية، تليها واحدة لإعادة كتابة مجموعة الملخصات الجزئية إلى ملخص كامل. باستخدام مجموعة بيانات ذات ضبط دقيقة تم اختيارها بعناية، تظهر هذه الطريقة فعالة في التعامل مع محادثات أطول، وتحسين جودة الملخصات التي تم إنشاؤها. نقوم بإجراء كل من التقييم التلقائي (من خلال Rouge ومقاييس مقرها المفهوم يركز على النتائج الطبية) وتقييم بشري (من خلال أمثلة نوعية من الأدبيات، تقييم الهلوسة، التعميم، الطلاقة، والنوعية العامة للملخصات التي تم إنشاؤها).
Fine-tuning pretrained models for automatically summarizing doctor-patient conversation transcripts presents many challenges: limited training data, significant domain shift, long and noisy transcripts, and high target summary variability. In this paper, we explore the feasibility of using pretrained transformer models for automatically summarizing doctor-patient conversations directly from transcripts. We show that fluent and adequate summaries can be generated with limited training data by fine-tuning BART on a specially constructed dataset. The resulting models greatly surpass the performance of an average human annotator and the quality of previous published work for the task. We evaluate multiple methods for handling long conversations, comparing them to the obvious baseline of truncating the conversation to fit the pretrained model length limit. We introduce a multistage approach that tackles the task by learning two fine-tuned models: one for summarizing conversation chunks into partial summaries, followed by one for rewriting the collection of partial summaries into a complete summary. Using a carefully chosen fine-tuning dataset, this method is shown to be effective at handling longer conversations, improving the quality of generated summaries. We conduct both an automatic evaluation (through ROUGE and two concept-based metrics focusing on medical findings) and a human evaluation (through qualitative examples from literature, assessing hallucination, generalization, fluency, and general quality of the generated summaries).
References used
https://aclanthology.org/
rate research
Read More
We focus on dialog models in the context of clinical studies where the goal is to help gather, in addition to the close information collected based on a questionnaire, serendipitous information that is medically relevant. To promote user engagement a
nd address this dual goal (collecting both a predefined set of data points and more informal information about the state of the patients), we introduce an ensemble model made of three bots: a task-based, a follow-up and a social bot. We introduce a generic method for developing follow-up bots. We compare different ensemble configurations and we show that the combination of the three bots (i) provides a better basis for collecting information than just the information seeking bot and (ii) collects information in a more user-friendly, more efficient manner that an ensemble model combining the information seeking and the social bot.
Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and i
nference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. We also conduct experiments on our newly proposed benchmark to show that the augmentation effect is not only attributed to memorization. Further ablation studies and a qualitative analysis provide more insights into our approach.
Patient safety is a modern but not new concept in global health care systems where reports and analyzes indicate that medical errors lead to adverse events. While the issue of safety in any health institution is a criterion in itself and a right of t
he patient, the importance of avoiding adverse patient events was not known until 1990, when astonishing numbers of statistical reports of multiple countries showed that patient morbidity and mortality had occurred Due to medical and nursing errors worldwide, where these statistics made their way to the public through the most famous statistical report prepared by the Institute of Medicine (IOM) and published in 1990, "To err is human, Kohn, Corrigan & Donaldson, 1990,". The main recommendations of the Medical Institute were to emphasize the need to adopt standards of practice and performance focused more on safety (patient safety).
Using data from English cloze tests, in which subjects also self-reported their gender, age, education, and race, we examine performance differences of pretrained language models across demographic groups, defined by these (protected) attributes. We
demonstrate wide performance gaps across demographic groups and show that pretrained language models systematically disfavor young non-white male speakers; i.e., not only do pretrained language models learn social biases (stereotypical associations) -- pretrained language models also learn sociolectal biases, learning to speak more like some than like others. We show, however, that, with the exception of BERT models, larger pretrained language models reduce some the performance gaps between majority and minority groups.
Existing work on probing of pretrained language models (LMs) has predominantly focused on sentence-level syntactic tasks. In this paper, we introduce document-level discourse probing to evaluate the ability of pretrained LMs to capture document-level
relations. We experiment with 7 pretrained LMs, 4 languages, and 7 discourse probing tasks, and find BART to be overall the best model at capturing discourse --- but only in its encoder, with BERT performing surprisingly well as the baseline model. Across the different models, there are substantial differences in which layers best capture discourse information, and large disparities between models.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
