Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEvaluating Document Coherence Modeling
تقييم النمذجة التماسك المستند
584
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
الملخص بينما طردت نماذج اللغة المحددة (LMS) مكاسب مثيرة للإعجاب على المهام المورفو والدلية، وقدرتها على نموذج الخطاب والظواهر البراغماتية أقل وضوحا.كخطوة نحو فهم أفضل لقدرات النمذجة خطابها، نقترح مهمة كشف التسلل.ندرس أداء مجموعة واسعة من LMS المحدد مسبقا على مهمة الكشف هذه للغة الإنجليزية.تفتقر إلى مجموعة بيانات للمهمة، ونحن نقدم DataSet Inteded Inteded، وهي عبارة عن بيانات الكشف عن عقوبة الدخيل الرواية، والتي تحتوي على 170،000+ مستندات مصنوعة من مقالات أخبار Wikipedia و CNN الإنجليزية.تظهر تجاربنا أن LMS المسبدة مسبقا تؤدي بشكل مثير للإعجاب في التقييم داخل المجال، بل تواجه انخفاضا كبيرا في إعداد المجال المتبادل، مما يشير إلى قدرة تعميم محدودة.نتائج أخرى على مجموعة بيانات مسبار لغوية جديدة تظهر أن هناك مجالا كبيرا للتحسين، خاصة في إعداد المجال المتقاطع.
Abstract While pretrained language models (LMs) have driven impressive gains over morpho-syntactic and semantic tasks, their ability to model discourse and pragmatic phenomena is less clear. As a step towards a better understanding of their discourse modeling capabilities, we propose a sentence intrusion detection task. We examine the performance of a broad range of pretrained LMs on this detection task for English. Lacking a dataset for the task, we introduce INSteD, a novel intruder sentence detection dataset, containing 170,000+ documents constructed from English Wikipedia and CNN news articles. Our experiments show that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity. Further results over a novel linguistic probe dataset show that there is substantial room for improvement, especially in the cross- domain setting.
References used
https://aclanthology.org/
rate research
Read More
The task of Event Detection (ED) in Information Extraction aims to recognize and classify trigger words of events in text. The recent progress has featured advanced transformer-based language models (e.g., BERT) as a critical component in state-of-th
e-art models for ED. However, the length limit for input texts is a barrier for such ED models as they cannot encode long-range document-level context that has been shown to be beneficial for ED. To address this issue, we propose a novel method to model document-level context for ED that dynamically selects relevant sentences in the document for the event prediction of the target sentence. The target sentence will be then augmented with the selected sentences and consumed entirely by transformer-based language models for improved representation learning for ED. To this end, the REINFORCE algorithm is employed to train the relevant sentence selection for ED. Several information types are then introduced to form the reward function for the training process, including ED performance, sentence similarity, and discourse relations. Our extensive experiments on multiple benchmark datasets reveal the effectiveness of the proposed model, leading to new state-of-the-art performance.
Natural language generation (NLG) tasks on pro-drop languages are known to suffer from zero pronoun (ZP) problems, and the problems remain challenging due to the scarcity of ZP-annotated NLG corpora. In this case, we propose a highly adaptive two-sta
ge approach to couple context modeling with ZP recovering to mitigate the ZP problem in NLG tasks. Notably, we frame the recovery process in a task-supervised fashion where the ZP representation recovering capability is learned during the NLG task learning process, thus our method does not require NLG corpora annotated with ZPs. For system enhancement, we learn an adversarial bot to adjust our model outputs to alleviate the error propagation caused by mis-recovered ZPs. Experiments on three document-level NLG tasks, i.e., machine translation, question answering, and summarization, show that our approach can improve the performance to a great extent, and the improvement on pronoun translation is very impressive.
Developers of text generation models rely on automated evaluation metrics as a stand-in for slow and expensive manual evaluations. However, image captioning metrics have struggled to give accurate learned estimates of the semantic and pragmatic succe
ss of output text. We address this weakness by introducing the first discourse-aware learned generation metric for evaluating image descriptions. Our approach is inspired by computational theories of discourse for capturing information goals using coherence. We present a dataset of image--description pairs annotated with coherence relations. We then train a coherence-aware metric on a subset of the Conceptual Captions dataset and measure its effectiveness---its ability to predict human ratings of output captions---on a test set composed of out-of-domain images. We demonstrate a higher Kendall Correlation Coefficient for our proposed metric with the human judgments for the results of a number of state-of-the-art coherence-aware caption generation models when compared to several other metrics including recently proposed learned metrics such as BLEURT and BERTScore.
A crucial difference between single- and multi-document summarization is how salient content manifests itself in the document(s). While such content may appear at the beginning of a single document, essential information is frequently reiterated in a
set of documents related to a particular topic, resulting in an endorsement effect that increases information salience. In this paper, we model the cross-document endorsement effect and its utilization in multiple document summarization. Our method generates a synopsis from each document, which serves as an endorser to identify salient content from other documents. Strongly endorsed text segments are used to enrich a neural encoder-decoder model to consolidate them into an abstractive summary. The method has a great potential to learn from fewer examples to identify salient content, which alleviates the need for costly retraining when the set of documents is dynamically adjusted. Through extensive experiments on benchmark multi-document summarization datasets, we demonstrate the effectiveness of our proposed method over strong published baselines. Finally, we shed light on future research directions and discuss broader challenges of this task using a case study.
The OCT is a recent diagnostic tool for non-invasive tissue examination, which has
been used in clinical practice since 1995. The OCT is similar to the ultrasonic tomography
in that it relies on optical waves rather than ultrasound ones. The abilit
y to obtain highcontrast
images of the retina in a non-invasive manner has made this examination very
important in retinal diseases associated with macula..
Therefore, our objective is to study the statistical values of the measurements of the
macular thickness using Optical Coherence Tomography (OCT) and the relationship of
each value to age, sex, medication ,duration of diabetes ,and the visual acuity in diabetic
eyes. Therefore, the accurate measurement of the macular thickness is important to follow
up disease progression, evaluate treatment and accurately diagnose lesions of visual
acuity.
The thickness of the macula in the left and right eye was thicker in males than in
females. We find out that the visual ability declines with age. For both left and right eyes,
the thickness of the macula increases with age and the content of the macula correlates
with its thickness and the period of the disease. There are no statistically significant
differences of the thicknesses with different treatment types (insulin or pills),so we notice a
correlation between the visual acuity and the thickness of the macula in both eyes ,so that
the higher the thickness of the macula in diabetic patients ,the lower the visual acuity . In
the study of the different parts of the macula, the internal part is thicker than the external
part and the nasal part is thicker than the different parts of the macula, and the nasointernal
part is the thickest.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
