Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCross-lingual Fine-tuning for Abstractive Arabic Text Summarization
تبادل اللغات الدقيق لتلصع النص الجماعي باللغة العربية
355
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في حين أن التلخيص المبشط في بعض اللغات، مثل اللغة الإنجليزية، فقد وصلت بالفعل نتائج جيدة إلى حد ما بسبب توفر موارد تحديد الاتجاه، مثل مجموعة بيانات CNN / Daily Mail، والتقدم الكبير في النماذج العصبية الإنتاجية، والتقدم المحرز في تلخيص الجماع للعربية اللغة الخامسة الأكثر تكلم على مستوى العالم، لا تزال في أحذية الأطفال. في حين أن بعض الموارد لتلخيص الاستخراجي كانت متاحة لبعض الوقت، في هذه الورقة، نقدم أول كائن من ملخصات الأخبار المذهلة التي كتبها الإنسان باللغة العربية، على أمل وضع أساس هذا الخط من البحث لهذه اللغة الهامة. تتكون DataSet من أكثر من 21 ألف عنصر. استخدمنا هذه البيانات هذه البيانات لتدريب مجموعة من أنظمة تلخيص الجماع العصبي للعربية من قبل طرازات اللغة المدربة مسبقا مسبقا مثل بيرت متعددة اللغات وأرابيرت والفتنة متعددة اللغات - 50. كما مجموعة البيانات العربية أصغر بكثير من .g. DataSet CNN / Daily Mail DataSet، كما طبقنا نقل المعرفة عبر اللغات إلى تحسين أداء أنظمةنا الأساسية. تضمنت الإعدادات نماذج ملخصتين تعتمد على M-Bert تدرب أصلا على الهنغارية / الإنجليزية ونظام مماثل على أساس M-BART-50 مدربا أصلا للروسية التي تم ضبطها بشكل جيد للعربية. تم إجراء تقييم النماذج من حيث الحمر، وتم تقييم يدوي للطلاقة وكفاية النماذج أيضا.
While abstractive summarization in certain languages, like English, has already reached fairly good results due to the availability of trend-setting resources, like the CNN/Daily Mail dataset, and considerable progress in generative neural models, progress in abstractive summarization for Arabic, the fifth most-spoken language globally, is still in baby shoes. While some resources for extractive summarization have been available for some time, in this paper, we present the first corpus of human-written abstractive news summaries in Arabic, hoping to lay the foundation of this line of research for this important language. The dataset consists of more than 21 thousand items. We used this dataset to train a set of neural abstractive summarization systems for Arabic by fine-tuning pre-trained language models such as multilingual BERT, AraBERT, and multilingual BART-50. As the Arabic dataset is much smaller than e.g. the CNN/Daily Mail dataset, we also applied cross-lingual knowledge transfer to significantly improve the performance of our baseline systems. The setups included two M-BERT-based summarization models originally trained for Hungarian/English and a similar system based on M-BART-50 originally trained for Russian that were further fine-tuned for Arabic. Evaluation of the models was performed in terms of ROUGE, and a manual evaluation of fluency and adequacy of the models was also performed.
References used
https://aclanthology.org/
rate research
Read More
Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. Their success, however, heavily depends on the availability of many annotated image-caption datasets for pretraining, where the t
exts are not necessarily in English. Although we can utilize machine translation (MT) tools to translate non-English text to English, the performance still largely relies on MT's quality and may suffer from high latency problems in real-world applications. This paper proposes a new approach to learn cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. We seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual and parallel corpus. We show that our approach achieves SOTA performance in retrieval tasks on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions.
In cross-lingual text classification, it is required that task-specific training data in high-resource source languages are available, where the task is identical to that of a low-resource target language. However, collecting such training data can b
e infeasible because of the labeling cost, task characteristics, and privacy concerns. This paper proposes an alternative solution that uses only task-independent word embeddings of high-resource languages and bilingual dictionaries. First, we construct a dictionary-based heterogeneous graph (DHG) from bilingual dictionaries. This opens the possibility to use graph neural networks for cross-lingual transfer. The remaining challenge is the heterogeneity of DHG because multiple languages are considered. To address this challenge, we propose dictionary-based heterogeneous graph neural network (DHGNet) that effectively handles the heterogeneity of DHG by two-step aggregations, which are word-level and language-level aggregations. Experimental results demonstrate that our method outperforms pretrained models even though it does not access to large corpora. Furthermore, it can perform well even though dictionaries contain many incorrect translations. Its robustness allows the usage of a wider range of dictionaries such as an automatically constructed dictionary and crowdsourced dictionary, which are convenient for real-world applications.
Automatic abstractive summaries are found to often distort or fabricate facts in the article. This inconsistency between summary and original text has seriously impacted its applicability. We propose a fact-aware summarization model FASum to extract
and integrate factual relations into the summary generation process via graph attention. We then design a factual corrector model FC to automatically correct factual errors from summaries generated by existing systems. Empirical results show that the fact-aware summarization can produce abstractive summaries with higher factual consistency compared with existing systems, and the correction model improves the factual consistency of given summaries via modifying only a few keywords.
Large scale pretrained models have demonstrated strong performances on several natural language generation and understanding benchmarks. However, introducing commonsense into them to generate more realistic text remains a challenge. Inspired from pre
vious work on commonsense knowledge generation and generative commonsense reasoning, we introduce two methods to add commonsense reasoning skills and knowledge into abstractive summarization models. Both methods beat the baseline on ROUGE scores, demonstrating the superiority of our models over the baseline. Human evaluation results suggest that summaries generated by our methods are more realistic and have fewer commonsensical errors.
The main purpose of the present research is to support Arabic Text- to - Speech synthesizers, with
natural prosody, based on linguistic analysis of texts to synthesize, and automatic prosody generation,
using rules which are deduced from recorded s
ignals analysis, of different types of sentences in Arabic. All
the types of Arabic sentences (declarative and constructive) were enumerated with the help of an expert in
Arabic linguistics . A textual corpus of about 2500 sentences covering most of these types was built and
recorded both in natural prosody and without prosody. Later, these sentences were analyzed to extract
prosody effect on the signal parameters, and to build prosody generation rules. In this paper, we present
the results on negation sentences, applied on synthesized speech using the open source tool MBROLA. The
results can be used with any parametric Arabic synthesizer. Future work will apply the rules on a new
Arabic synthesizer based on semi-syllables units, which is under development in the Higher Institute for
Applied Sciences and Technology.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
