منذ أن تم اعتماد النماذج العصبية في توليد لغة البيانات إلى النص، فقد تم اعتمادها دائما على المكونات الخارجية لتحسين دقتها الدلالية، لأن النماذج عادة لا تظهر القدرة على توليد نص يذكر بشكل موثوق كل المعلومات المقدمة فيالمدخل.في هذه الورقة، نقترح طريقة فك التشفير الجديدة التي تستخرج معلومات تفسيرها من نماذج ترميز تشفير التشفير، وتستخدمها لاستنتاج السمات التي يتم ذكرها في النص الذي تم إنشاؤه، والذي يستخدم لاحقا لإنقاش فرضيات شعاع.باستخدام طريقة فك التشفير هذه مع T5 و Bart، نعرض على ثلاثة مجموعات بيانات قدرتها على تقليل الأخطاء الدلالية بشكل كبير في المخرجات التي تم إنشاؤها، مع الحفاظ على جودة حديثة من بين الفن.
Ever since neural models were adopted in data-to-text language generation, they have invariably been reliant on extrinsic components to improve their semantic accuracy, because the models normally do not exhibit the ability to generate text that reliably mentions all of the information provided in the input. In this paper, we propose a novel decoding method that extracts interpretable information from encoder-decoder models' cross-attention, and uses it to infer which attributes are mentioned in the generated text, which is subsequently used to rescore beam hypotheses. Using this decoding method with T5 and BART, we show on three datasets its ability to dramatically reduce semantic errors in the generated outputs, while maintaining their state-of-the-art quality.
References used
https://aclanthology.org/
We study the power of cross-attention in the Transformer architecture within the context of transfer learning for machine translation, and extend the findings of studies into cross-attention when training from scratch. We conduct a series of experime
In this paper, we describe our system used for SemEval 2021 Task 7: HaHackathon: Detecting and Rating Humor and Offense. We used a simple fine-tuning approach using different Pre-trained Language Models (PLMs) to evaluate their performance for humor
This paper presents our system for the Quantity span identification, Unit of measurement identification and Value modifier classification subtasks of the MeasEval 2021 task. The purpose of the Quantity span identification task was to locate spans of
This paper discusses different approaches to the Toxic Spans Detection task. The problem posed by the task was to determine which words contribute mostly to recognising a document as toxic. As opposed to binary classification of entire texts, word-le
The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters. However, due to deployment constraints in edge devices, there has been a rising interest in the compression o