نقترح طريقة لتعلم تمثيلات الجملة المعممة والتعميم باستخدام التعلم المشروع للإشراف على الذات.في الطريقة المقترحة، يتم إعطاء نموذج نص يتكون من جمل متعددة.تم اختيار جملة واحدة بشكل عشوائي كجوزة مستهدفة.يتم تدريب النموذج على زيادة التشابه بين تمثيل الجملة المستهدفة مع سياقها وذلك من الجملة المستهدفة الملثملة بنفس السياق.في الوقت نفسه، يقلل النموذج من التشابه بين التمثيل الأخير وتمثيل جملة عشوائية مع نفس السياق.نحن نطبق طريقنا لتحليل علاقة الخطاب باللغة الإنجليزية واليابانية وإظهار أنه يتفوق على أساليب خطية قوية على أساس Bert و Xlnet و Roberta.
We propose a method to learn contextualized and generalized sentence representations using contrastive self-supervised learning. In the proposed method, a model is given a text consisting of multiple sentences. One sentence is randomly selected as a target sentence. The model is trained to maximize the similarity between the representation of the target sentence with its context and that of the masked target sentence with the same context. Simultaneously, the model minimizes the similarity between the latter representation and the representation of a random sentence with the same context. We apply our method to discourse relation analysis in English and Japanese and show that it outperforms strong baseline methods based on BERT, XLNet, and RoBERTa.
References used
https://aclanthology.org/
Spoken question answering (SQA) requires fine-grained understanding of both spoken documents and questions for the optimal answer prediction. In this paper, we propose novel training schemes for spoken question answering with a self-supervised traini
Politicians often have underlying agendas when reacting to events. Arguments in contexts of various events reflect a fairly consistent set of agendas for a given entity. In spite of recent advances in Pretrained Language Models, those text representa
Common acquisition functions for active learning use either uncertainty or diversity sampling, aiming to select difficult and diverse data points from the pool of unlabeled data, respectively. In this work, leveraging the best of both worlds, we prop
Learning sentence embeddings from dialogues has drawn increasing attention due to its low annotation cost and high domain adaptability. Conventional approaches employ the siamese-network for this task, which obtains the sentence embeddings through mo
Discourse parsers recognize the intentional and inferential relationships that organize extended texts. They have had a great influence on a variety of NLP tasks as well as theoretical studies in linguistics and cognitive science. However it is often