Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userNeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
فك التشفير العصبي: (الأمم المتحدة) يشرف على توليد النص العصبي مع قيود المنطق المسند
252
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
غالبا ما يتطلب جيل النص الشرطي القيود المعجمية، أي الكلمات التي يجب أو لا ينبغي إدراجها في نص الإخراج. في حين أن الوصفة المهيمنة لجيل النظام الشرطي كانت نماذج لغوية متماثلة على نطاق واسع يتم تصويرها على بيانات التدريب الخاصة بمهام المهام، فإن مثل هذه النماذج لا تتعلم اتباع القيود الأساسية بشكل موثوق، حتى عند الإشراف على كميات كبيرة من الأمثلة الخاصة بمهام المهام وبعد نقترح فك التشفير العصبي، خوارزمية بسيطة ولكنها فعالة تمكن نماذج اللغة العصبية - تحت إشراف أو لا - لتوليد نص بطلاقة مع مرضية القيود المعقدة المعقدة. نهجنا قوي بعد كفاءة. يتعامل مع أي مجموعة من القيود المعجمية المعبرة تحت المنطق المسند، في حين أن وقت التشغيل مقاربها يعادل البحث عن شعاع التقليدية. تظهر النتائج التجريبية على أربعة معايير أن فك التشفير العصبي تتفوق على النهج السابقة، بما في ذلك الخوارزميات التي تتعامل مع مجموعة فرعية من قيودنا. علاوة على ذلك، نجد أن النماذج غير الخاضعة للكشف عن فك التشفير العصبي في كثير من الأحيان تفوق النماذج الخاضعة للإشراف مع فك التشفير التقليدي، حتى عندما تستند الأخير إلى شبكات أكبر بكثير. تشير نتائجنا إلى حد الشبكات العصبية واسعة النطاق لتوليد القابل للتحكم بالقلق ووعد خوارزميات وقت الاستقدمية.
Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. While the dominant recipe for conditional text generation has been large-scale pretrained language models that are finetuned on the task-specific training data, such models do not learn to follow the underlying constraints reliably, even when supervised with large amounts of task-specific examples. We propose NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models -- supervised or not -- to generate fluent text while satisfying complex lexical constraints. Our approach is powerful yet efficient. It handles any set of lexical constraints that is expressible under predicate logic, while its asymptotic runtime is equivalent to conventional beam search. Empirical results on four benchmarks show that NeuroLogic Decoding outperforms previous approaches, including algorithms that handle a subset of our constraints. Moreover, we find that unsupervised models with NeuroLogic Decoding often outperform supervised models with conventional decoding, even when the latter is based on considerably larger networks. Our results suggest the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.
References used
https://aclanthology.org/
rate research
Read More
Narrative generation is an open-ended NLP task in which a model generates a story given a prompt. The task is similar to neural response generation for chatbots; however, innovations in response generation are often not applied to narrative generatio
n, despite the similarity between these tasks. We aim to bridge this gap by applying and evaluating advances in decoding methods for neural response generation to neural narrative generation. In particular, we employ GPT-2 and perform ablations across nucleus sampling thresholds and diverse decoding hyperparameters---specifically, maximum mutual information---analyzing results over multiple criteria with automatic and human evaluation. We find that (1) nucleus sampling is generally best with thresholds between 0.7 and 0.9; (2) a maximum mutual information objective can improve the quality of generated stories; and (3) established automatic metrics do not correlate well with human judgments of narrative quality on any qualitative metric.
Knowledge graphs (KG) have become increasingly important to endow modern recommender systems with the ability to generate traceable reasoning paths to explain the recommendation process. However, prior research rarely considers the faithfulness of th
e derived explanations to justify the decision-making process. To the best of our knowledge, this is the first work that models and evaluates faithfully explainable recommendation under the framework of KG reasoning. Specifically, we propose neural logic reasoning for explainable recommendation (LOGER) by drawing on interpretable logical rules to guide the path-reasoning process for explanation generation. We experiment on three large-scale datasets in the e-commerce domain, demonstrating the effectiveness of our method in delivering high-quality recommendations as well as ascertaining the faithfulness of the derived explanation.
The shift to neural models in Referring Expression Generation (REG) has enabled more natural set-ups, but at the cost of interpretability. We argue that integrating pragmatic reasoning into the inference of context-agnostic generation models could re
concile traits of traditional and neural REG, as this offers a separation between context-independent, literal information and pragmatic adaptation to context. With this in mind, we apply existing decoding strategies from discriminative image captioning to REG and evaluate them in terms of pragmatic informativity, likelihood to ground-truth annotations and linguistic diversity. Our results show general effectiveness, but a relatively small gain in informativity, raising important questions for REG in general.
We propose a shared task on training instance selection for few-shot neural text generation. Large-scale pretrained language models have led to dramatic improvements in few-shot text generation. Nonetheless, almost all previous work simply applies ra
ndom sampling to select the few-shot training instances. Little to no attention has been paid to the selection strategies and how they would affect model performance. Studying the selection strategy can help us (1) make the most use of our annotation budget in downstream tasks and (2) better benchmark few-shot text generative models. We welcome submissions that present their selection strategies and the effects on the generation quality.
Most current neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to-left. In this work, we propose a novel method that breaks up the limitation of these decoding orders, called Smart-Start decoding. Mor
e specifically, our method first predicts a median word. It starts to decode the words on the right side of the median word and then generates words on the left. We evaluate the proposed Smart-Start decoding method on three datasets. Experimental results show that the proposed method can significantly outperform strong baseline models.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
