لالتقاط بنية الرسم البياني الدلالي من النص الخام، يتم بناء معظم طرق التلخيص الموجودة على GNNS مع نموذج مدرب مسبقا.ومع ذلك، فإن هذه الأساليب تعاني من إجراءات مرهقة وحسابات غير فعالة وثائق نصية طويلة.لتخفيف هذه المشكلات، تقترح هذه الورقة HETFORMER، وهو نموذج مدرب مسبقا من المحولات مع انتباه متفرج متعدد الحبيبات لتلخيص الاستخراجي لفترة طويلة.على وجه التحديد، نقوم بالنماذج أنواع مختلفة من العقد الدلالية في النص الخام كشركة بيانية غير متجانسة محتملة وتعلم العلاقات المخللة بشكل مباشر (حواف) بين العقد بواسطة المحول.تظهر تجارب واسعة النطاق على كل من مهام تلخيص مستندات واحدة ومتعددة المستندات أن HETFORMER تحقق أداء حديثة في Rouge F1 أثناء استخدام ذاكرة أقل ومعلمات أقل.
To capture the semantic graph structure from raw text, most existing summarization approaches are built on GNNs with a pre-trained model. However, these methods suffer from cumbersome procedures and inefficient computations for long-text documents. To mitigate these issues, this paper proposes HetFormer, a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization. Specifically, we model different types of semantic nodes in raw text as a potential heterogeneous graph and directly learn heterogeneous relationships (edges) among nodes by Transformer. Extensive experiments on both single- and multi-document summarization tasks show that HetFormer achieves state-of-the-art performance in Rouge F1 while using less memory and fewer parameters.
References used
https://aclanthology.org/
Despite the recent advances in applying pre-trained language models to generate high-quality texts, generating long passages that maintain long-range coherence is yet challenging for these models. In this paper, we propose DiscoDVT, a discourse-aware
Extractive text summarization aims at extracting the most representative sentences from a given document as its summary. To extract a good summary from a long text document, sentence embedding plays an important role. Recent studies have leveraged gr
Neural-based summarization models suffer from the length limitation of text encoder. Long documents have to been truncated before they are sent to the model, which results in huge loss of summary-relevant contents. To address this issue, we propose t
Coupled with biaffine decoders, transformers have been effectively adapted to text-to-graph transduction and achieved state-of-the-art performance on AMR parsing. Many prior works, however, rely on the biaffine decoder for either or both arc and labe
Automatic summarization aims to extract important information from large amounts of textual data in order to create a shorter version of the original texts while preserving its information. Training traditional extractive summarization models relies