Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDecoding Methods for Neural Narrative Generation
طرق فك تشفير الجيل السرد العصبي
347
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
الجيل السردي هو مهمة NLP مفتوحة العضوية التي يولد فيها نموذج قصة إعطاء موجه.المهمة تشبه توليد الاستجابة العصبية لل Chatbots؛ومع ذلك، غالبا ما لا يتم تطبيق الابتكارات في توليد الاستجابة على جيل سرد، على الرغم من التشابه بين هذه المهام.نحن نهدف إلى سد هذه الفجوة من خلال تطبيق وتقييم التقدم في طرق فك تشفير جيل الاستجابة العصبية إلى توليد السرد العصبي.على وجه الخصوص، نحن نوظف GPT-2 وأداء الأزمة عبر عتبات أخذ العينات النواة ومثبتة تنوعا فرطيا مثبطا --- على وجه التحديد، والحد الأقصى للمعلومات المتبادلة - - تحليل النتائج على معايير متعددة مع التقييم التلقائي والإنساني.نجد أن (1) أخذ عينات نواة أفضل عموما مع عتبات بين 0.7 و 0.9؛(2) الحد الأقصى لهدف المعلومات المتبادلة يمكن أن يحسن نوعية القصص التي تم إنشاؤها؛و (3) لا ترتبط مقاييس التلقائية المنشأة بشكل جيد مع الأحكام الإنسانية لجودة السرد على أي متري نوعي.
Narrative generation is an open-ended NLP task in which a model generates a story given a prompt. The task is similar to neural response generation for chatbots; however, innovations in response generation are often not applied to narrative generation, despite the similarity between these tasks. We aim to bridge this gap by applying and evaluating advances in decoding methods for neural response generation to neural narrative generation. In particular, we employ GPT-2 and perform ablations across nucleus sampling thresholds and diverse decoding hyperparameters---specifically, maximum mutual information---analyzing results over multiple criteria with automatic and human evaluation. We find that (1) nucleus sampling is generally best with thresholds between 0.7 and 0.9; (2) a maximum mutual information objective can improve the quality of generated stories; and (3) established automatic metrics do not correlate well with human judgments of narrative quality on any qualitative metric.
References used
https://aclanthology.org/
rate research
Read More
Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. While the dominant recipe for conditional text generation has been large-scale pretrained language models that are f
inetuned on the task-specific training data, such models do not learn to follow the underlying constraints reliably, even when supervised with large amounts of task-specific examples. We propose NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models -- supervised or not -- to generate fluent text while satisfying complex lexical constraints. Our approach is powerful yet efficient. It handles any set of lexical constraints that is expressible under predicate logic, while its asymptotic runtime is equivalent to conventional beam search. Empirical results on four benchmarks show that NeuroLogic Decoding outperforms previous approaches, including algorithms that handle a subset of our constraints. Moreover, we find that unsupervised models with NeuroLogic Decoding often outperform supervised models with conventional decoding, even when the latter is based on considerably larger networks. Our results suggest the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.
Understanding speaker's feelings and producing appropriate responses with emotion connection is a key communicative skill for empathetic dialogue systems. In this paper, we propose a simple technique called Affective Decoding for empathetic response
generation. Our method can effectively incorporate emotion signals during each decoding step, and can additionally be augmented with an auxiliary dual emotion encoder, which learns separate embeddings for the speaker and listener given the emotion base of the dialogue. Extensive empirical studies show that our models are perceived to be more empathetic by human evaluations, in comparison to several strong mainstream methods for empathetic responding.
Most current neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to-left. In this work, we propose a novel method that breaks up the limitation of these decoding orders, called Smart-Start decoding. Mor
e specifically, our method first predicts a median word. It starts to decode the words on the right side of the median word and then generates words on the left. We evaluate the proposed Smart-Start decoding method on three datasets. Experimental results show that the proposed method can significantly outperform strong baseline models.
Over the past decade, the field of natural language processing has developed a wide array of computational methods for reasoning about narrative, including summarization, commonsense inference, and event detection. While this work has brought an impo
rtant empirical lens for examining narrative, it is by and large divorced from the large body of theoretical work on narrative within the humanities, social and cognitive sciences. In this position paper, we introduce the dominant theoretical frameworks to the NLP community, situate current research in NLP within distinct narratological traditions, and argue that linking computational work in NLP to theory opens up a range of new empirical questions that would both help advance our understanding of narrative and open up new practical applications.
The shift to neural models in Referring Expression Generation (REG) has enabled more natural set-ups, but at the cost of interpretability. We argue that integrating pragmatic reasoning into the inference of context-agnostic generation models could re
concile traits of traditional and neural REG, as this offers a separation between context-independent, literal information and pragmatic adaptation to context. With this in mind, we apply existing decoding strategies from discriminative image captioning to REG and evaluate them in terms of pragmatic informativity, likelihood to ground-truth annotations and linguistic diversity. Our results show general effectiveness, but a relatively small gain in informativity, raising important questions for REG in general.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
