Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userWhat Would it Take to get Biomedical QA Systems into Practice?
ماذا سيستغرق الحصول على أنظمة ضمان الجودة الطبية الحيوية موضع التنفيذ؟
617
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
إن أنظمة الإجابة على السؤال الطبي (QA) لديها القدرة على الرد على أوجه عدم اليقين للأطباء حول العلاج والتشخيص عند الطلب، على علم بأحدث الأدلة.ومع ذلك، على الرغم من التقدم الكبير في عام ضمان الجودة التي أدلى بها مجتمع NLP، لا تزال أنظمة ضمان الجودة الطبية تستخدم على نطاق واسع في البيئات السريرية.أحد الأسباب المحتملة لهذا هو أن الأطباء قد لا يرجعون بسهولة مخرجات نظام ضمان الجودة، جزئيا لأن الشفافية والثقة بالثقة، ولم تكن الأصل اعتبارات رئيسية في تصميم هذه النماذج.في هذه الورقة، نناقش مجموعة من المعايير التي، إذا التقت، فمن المرجح أن تزيد من المرجح أن تزيد من فائدة أنظمة ضمان الجودة الطبية الحيوية، والتي قد تؤدي بدورها إلى اعتماد هذه النظم في الممارسة العملية.نقوم بتقييم النماذج والمهام والموادات الموجودة فيما يتعلق بهذه المعايير وتسليط الضوء على أوجه القصور من الأساليب المقترحة سابقا والشديد نحو ما قد يكون أكثر قدرة للاستخدام من أنظمة ضمان الجودة.
Medical question answering (QA) systems have the potential to answer clinicians' uncertainties about treatment and diagnosis on-demand, informed by the latest evidence. However, despite the significant progress in general QA made by the NLP community, medical QA systems are still not widely used in clinical environments. One likely reason for this is that clinicians may not readily trust QA system outputs, in part because transparency, trustworthiness, and provenance have not been key considerations in the design of such models. In this paper we discuss a set of criteria that, if met, we argue would likely increase the utility of biomedical QA systems, which may in turn lead to adoption of such systems in practice. We assess existing models, tasks, and datasets with respect to these criteria, highlighting shortcomings of previously proposed approaches and pointing toward what might be more usable QA systems.
References used
https://aclanthology.org/
rate research
Read More
Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and biased systems score so highly on standard benchmarks that there is little room for researchers who develop better systems to demonstrate their improvements. The
recent trend to abandon IID benchmarks in favor of adversarially-constructed, out-of-distribution test sets ensures that current models will perform poorly, but ultimately only obscures the abilities that we want our benchmarks to measure. In this position paper, we lay out four criteria that we argue NLU benchmarks should meet. We argue most current benchmarks fail at these criteria, and that adversarial data collection does not meaningfully address the causes of these failures. Instead, restoring a healthy evaluation ecosystem will require significant progress in the design of benchmark datasets, the reliability with which they are annotated, their size, and the ways they handle social bias.
Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. In this paper, we proposed Mixture-of-Partitions (MoP), an infusion approach that can handle a very large knowledge graph (KG) by partitioning it in
to smaller sub-graphs and infusing their specific knowledge into various BERT models using lightweight adapters. To leverage the overall factual knowledge for a target task, these sub-graph adapters are further fine-tuned along with the underlying BERT through a mixture layer. We evaluate our MoP with three biomedical BERTs (SciBERT, BioBERT, PubmedBERT) on six downstream tasks (inc. NLI, QA, Classification), and the results show that our MoP consistently enhances the underlying BERTs in task performance, and achieves new SOTA performances on five evaluated datasets.
This paper describes the Tencent AI Lab submission of the WMT2021 shared task on biomedical translation in eight language directions: English-German, English-French, English-Spanish and English-Russian. We utilized different Transformer architectures
, pretraining and back-translation strategies to improve translation quality. Concretely, we explore mBART (Liu et al., 2020) to demonstrate the effectiveness of the pretraining strategy. Our submissions (Tencent AI Lab Machine Translation, TMT) in German/French/Spanish⇒English are ranked 1st respectively according to the official evaluation results in terms of BLEU scores.
When a model attribution technique highlights a particular part of the input, a user might understand this highlight as making a statement about counterfactuals (Miller, 2019): if that part of the input were to change, the model's prediction might ch
ange as well. This paper investigates how well different attribution techniques align with this assumption on realistic counterfactuals in the case of reading comprehension (RC). RC is a particularly challenging test case, as token-level attributions that have been extensively studied in other NLP tasks such as sentiment analysis are less suitable to represent the reasoning that RC models perform. We construct counterfactual sets for three different RC settings, and through heuristics that can connect attribution methods' outputs to high-level model behavior, we can evaluate how useful different attribution methods and even different formats are for understanding counterfactuals. We find that pairwise attributions are better suited to RC than token-level attributions across these different RC settings, with our best performance coming from a modification that we propose to an existing pairwise attribution method.
This paper describes the submission of Huawei Translation Service Center (HW-TSC) to WMT21 biomedical translation task in two language pairs: Chinese↔English and German↔English (Our registered team name is HuaweiTSC). Technical details are introduced
in this paper, including model framework, data pre-processing method and model enhancement strategies. In addition, using the wmt20 OK-aligned biomedical test set, we compare and analyze system performances under different strategies. On WMT21 biomedical translation task, Our systems in English→Chinese and English→German directions get the highest BLEU scores among all submissions according to the official evaluation results.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
