Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userNot Just Classification: Recognizing Implicit Discourse Relation on Joint Modeling of Classification and Generation
ليس فقط التصنيف: التعرف على علاقة خطاب ضمني على النمذجة المشتركة للتصنيف والجيل
373
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
التعرف على علاقة الخطاب الضمني (IDRR) هو مهمة حاسمة في تحليل الخطاب. الدراسات السابقة فقط اعتبارها مهمة التصنيف وتفتقر إلى فهم متعمق لدل العلاقات المختلفة. لذلك، نرى أولا EDRR كامرأة توليد ومزيد من اقتراح طريقة النمذجة المشتركة للتصنيف والجيل. على وجه التحديد، نقترح نموذجا مشتركا، CG-T5، للتعرف على تسمية العلاقة وتوليد الجملة المستهدفة التي تحتوي على معنى العلاقات في وقت واحد. علاوة على ذلك، نقوم بتصميم ثلاث نماذج جملة مستهدفة، بما في ذلك نموذج الأسئلة، لنموذج الجيل لإدماج المعرفة السابقة. لمعالجة مشكلة أن وحدات الخطاب الكبيرة غير متضمنة بالكاد في الجملة المستهدفة، نقترح أيضا آلية بناء الجملة المستهدفة التي تستخرج الجمل الأساسية تلقائيا من تلك الوحدات الخطابية الكبيرة. تظهر النتائج التجريبية على حد سواء على مجموعات بيانات MCDTB والإنجليزية الصينية أن نموذج CG-T5 لدينا يحقق أفضل أداء ضد العديد من الأنظمة الحديثة.
Implicit discourse relation recognition (IDRR) is a critical task in discourse analysis. Previous studies only regard it as a classification task and lack an in-depth understanding of the semantics of different relations. Therefore, we first view IDRR as a generation task and further propose a method joint modeling of the classification and generation. Specifically, we propose a joint model, CG-T5, to recognize the relation label and generate the target sentence containing the meaning of relations simultaneously. Furthermore, we design three target sentence forms, including the question form, for the generation model to incorporate prior knowledge. To address the issue that large discourse units are hardly embedded into the target sentence, we also propose a target sentence construction mechanism that automatically extracts core sentences from those large discourse units. Experimental results both on Chinese MCDTB and English PDTB datasets show that our model CG-T5 achieves the best performance against several state-of-the-art systems.
References used
https://aclanthology.org/
rate research
Read More
This paper describes our submission to theSemEval'21: Task 7- HaHackathon: Detecting and Rating Humor and Offense. In this challenge, we explore intermediate finetuning, backtranslation augmentation, multitask learning, and ensembling of different la
nguage models. Curiously, intermediate finetuning and backtranslation do not improve performance, while multitask learning and ensembling do improve performance. We explore why intermediate finetuning and backtranslation do not provide the same benefit as other natural language processing tasks and offer insight into the errors that our model makes. Our best performing system ranks 7th on Task 1bwith an RMSE of 0.5339
Implicit discourse relation recognition (IDRR) aims to identify logical relations between two adjacent sentences in the discourse. Existing models fail to fully utilize the contextual information which plays an important role in interpreting each loc
al sentence. In this paper, we thus propose a novel graph-based Context Tracking Network (CT-Net) to model the discourse context for IDRR. The CT-Net firstly converts the discourse into the paragraph association graph (PAG), where each sentence tracks their closely related context from the intricate discourse through different types of edges. Then, the CT-Net extracts contextual representation from the PAG through a specially designed cross-grained updating mechanism, which can effectively integrate both sentence-level and token-level contextual semantics. Experiments on PDTB 2.0 show that the CT-Net gains better performance than models that roughly model the context.
Discourse parsers recognize the intentional and inferential relationships that organize extended texts. They have had a great influence on a variety of NLP tasks as well as theoretical studies in linguistics and cognitive science. However it is often
difficult to achieve good results from current discourse models, largely due to the difficulty of the task, particularly recognizing implicit discourse relations. Recent developments in transformer-based models have shown great promise on these analyses, but challenges still remain. We present a position paper which provides a systematic analysis of the state of the art discourse parsers. We aim to examine the performance of current discourse parsing models via gradual domain shift: within the corpus, on in-domain texts, and on out-of-domain texts, and discuss the differences between the transformer-based models and the previous models in predicting different types of implicit relations both inter- and intra-sentential. We conclude by describing several shortcomings of the existing models and a discussion of how future work should approach this problem.
In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. This is challenging even for humans, leading to shortage of annotated data, a fact that makes
the task even more difficult for supervised machine learning approaches. In the current study, we perform implicit discourse relation classification without relying on any labeled implicit relation. We sidestep the lack of data through explicitation of implicit relations to reduce the task to two sub-problems: language modeling and explicit discourse relation classification, a much easier problem. Our experimental results show that this method can even marginally outperform the state-of-the-art, in spite of being much simpler than alternative models of comparable performance. Moreover, we show that the achieved performance is robust across domains as suggested by the zero-shot experiments on a completely different domain. This indicates that recent advances in language modeling have made language models sufficiently good at capturing inter-sentence relations without the help of explicit discourse markers.
In natural language generation tasks, a neural language model is used for generating a sequence of words forming a sentence. The topmost weight matrix of the language model, known as the classification layer, can be viewed as a set of vectors, each r
epresenting a target word from the target dictionary. The target word vectors, along with the rest of the model parameters, are learned and updated during training. In this paper, we analyze the properties encoded in the target vectors and question the necessity of learning these vectors. We suggest to randomly draw the target vectors and set them as fixed so that no weights updates are being made during training. We show that by excluding the vectors from the optimization, the number of parameters drastically decreases with a marginal effect on the performance. We demonstrate the effectiveness of our method in image-captioning and machine-translation.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
