اجتذبت التعلم الذاتي الإشراف مؤخرا اهتماما كبيرا في مجتمع NLP لقدرته على تعلم الميزات التمييزية باستخدام هدف بسيط.تحقق هذه الورقة التي تحقق ما إذا كان التعلم مناقصة يمكن تمديده لإيلاء اهتمام Transfomer لمعالجة تحدي مخطط Winograd.تحقيقا لهذه الغاية، نقترح إطارا جديدا للإشراف على الذات، حيث يستحق خسارة صغيرة مباشرة على مستوى اهتمام الذات.يوضح التحليل التجريبي للنماذج التي تعتمد انتباهنا على مجموعات بيانات متعددة إمكانيات التفكير في المنطقية.يتفوق النهج المقترح على جميع النهج القابلة للمقارنة غير الخاضعة للرقابة مع تجاوز الأشرار في بعض الأحيان.
Self-supervised learning has recently attracted considerable attention in the NLP community for its ability to learn discriminative features using a contrastive objective. This paper investigates whether contrastive learning can be extended to Transfomer attention to tackling the Winograd Schema Challenge. To this end, we propose a novel self-supervised framework, leveraging a contrastive loss directly at the level of self-attention. Experimental analysis of our attention-based models on multiple datasets demonstrates superior commonsense reasoning capabilities. The proposed approach outperforms all comparable unsupervised approaches while occasionally surpassing supervised ones.
References used
https://aclanthology.org/
Learning sentence embeddings from dialogues has drawn increasing attention due to its low annotation cost and high domain adaptability. Conventional approaches employ the siamese-network for this task, which obtains the sentence embeddings through mo
Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. In this paper, we propose a new method with the goal
Large-scale auto-regressive models have achieved great success in dialogue response generation, with the help of Transformer layers. However, these models do not learn a representative latent space of the sentence distribution, making it hard to cont
Context-aware neural machine translation (NMT) incorporates contextual information of surrounding texts, that can improve the translation quality of document-level machine translation. Many existing works on context-aware NMT have focused on developi
Within the last few years, the number of Arabic internet users and Arabic online content is in exponential growth. Dealing with Arabic datasets and the usage of non-explicit sentences to express an opinion are considered to be the major challenges in