Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userConRPG: Paraphrase Generation using Contexts as Regularizer
conrpg: إعادة صياغة التوليد باستخدام السياقات كما العظمية
436
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
مشكلة طويلة الأمد مع إعادة صياغة إعادة صياغة هي الافتقار إلى إشارات الإشراف الموثوقة. في هذه الورقة، نقترح نموذجا جديدا غير منشئين من أجل إعادة صياغة إعادة صياغة إعادة صياغة نصها بناء على افتراض أن احتمالات توليد جملتين بنفس المعنى بالنظر إلى نفس السياق يجب أن يكون هو نفسه. مستوحاة من هذه الفكرة الأساسية، نقترح نظام خط أنابيب يتكون من إعادة صياغة ترجمة المرشحة بناء على نماذج اللغة السياقية، ومرشح المرشح باستخدام وظائف التسجيل، وإعادة صياغة التدريب النموذجي على أساس المرشحين المحددين. تقدم النموذج المقترح مزايا حول طرق إنشاء الصياغة الحالية: (1) باستخدام السياق العدواني على المعاني، يكون النموذج من إنشاء كميات ضخمة من أزواج إعادة صياغة عالية الجودة؛ (2) مزيج من المبلغ الهائل من المرشحين لإعادة صياغة الصياغة وزيادة ترويج التنوع يتجاوزون الخلاصات مع مزيد من التنوع المعجمي والنزاع؛ و (3) استخدام وظائف تسجيل التخصيب البشرية لتحديد إعادة صياغة أزواج من المرشحين، يوفر الإطار المقترح قناة للمطورين للتدخل مع عملية توليد البيانات، مما يؤدي إلى نموذج أكثر قابلية للتحكم. توضح النتائج التجريبية عبر المهام المختلفة ومجموعات البيانات أن النموذج المقترح تتفوق بشكل كبير على النهج التي يتم إعادة صياغة النصوص الموجودة في كل من الإعدادات الإشرافية وغير المدفوعة.
A long-standing issue with paraphrase generation is the lack of reliable supervision signals. In this paper, we propose a new unsupervised paradigm for paraphrase generation based on the assumption that the probabilities of generating two sentences with the same meaning given the same context should be the same. Inspired by this fundamental idea, we propose a pipelined system which consists of paraphrase candidate generation based on contextual language models, candidate filtering using scoring functions, and paraphrase model training based on the selected candidates. The proposed paradigm offers merits over existing paraphrase generation methods: (1) using the context regularizer on meanings, the model is able to generate massive amounts of high-quality paraphrase pairs; (2) the combination of the huge amount of paraphrase candidates and further diversity-promoting filtering yields paraphrases with more lexical and syntactic diversity; and (3) using human-interpretable scoring functions to select paraphrase pairs from candidates, the proposed framework provides a channel for developers to intervene with the data generation process, leading to a more controllable model. Experimental results across different tasks and datasets demonstrate that the proposed paradigm significantly outperforms existing paraphrase approaches in both supervised and unsupervised setups.
References used
https://aclanthology.org/
rate research
Read More
Paraphrase generation is an important task in natural language processing. Previous works focus on sentence-level paraphrase generation, while ignoring document-level paraphrase generation, which is a more challenging and valuable task. In this paper
, we explore the task of document-level paraphrase generation for the first time and focus on the inter-sentence diversity by considering sentence rewriting and reordering. We propose CoRPG (Coherence Relationship guided Paraphrase Generation), which leverages graph GRU to encode the coherence relationship graph and get the coherence-aware representation for each sentence, which can be used for re-arranging the multiple (possibly modified) input sentences. We create a pseudo document-level paraphrase dataset for training CoRPG. Automatic evaluation results show CoRPG outperforms several strong baseline models on the BERTScore and diversity scores. Human evaluation also shows our model can generate document paraphrase with more diversity and semantic preservation.
This paper focuses on paraphrase generation,which is a widely studied natural language generation task in NLP. With the development of neural models, paraphrase generation research has exhibited a gradual shift to neural methods in the recent years.
This has provided architectures for contextualized representation of an input text and generating fluent, diverseand human-like paraphrases. This paper surveys various approaches to paraphrase generation with a main focus on neural methods.
We propose to control paraphrase generation through carefully chosen target syntactic structures to generate more proper and higher quality paraphrases. Our model, AESOP, leverages a pretrained language model and adds deliberately chosen syntactical
control via a retrieval-based selection module to generate fluent paraphrases. Experiments show that AESOP achieves state-of-the-art performances on semantic preservation and syntactic conformation on two benchmark datasets with ground-truth syntactic control from human-annotated exemplars. Moreover, with the retrieval-based target syntax selection module, AESOP generates paraphrases with even better qualities than the current best model using human-annotated target syntactic parses according to human evaluation. We further demonstrate the effectiveness of AESOP to improve classification models' robustness to syntactic perturbation by data augmentation on two GLUE tasks.
Paraphrase generation is a longstanding NLP task that has diverse applications on downstream NLP tasks. However, the effectiveness of existing efforts predominantly relies on large amounts of golden labeled data. Though unsupervised endeavors have be
en proposed to alleviate this issue, they may fail to generate meaningful paraphrases due to the lack of supervision signals. In this work, we go beyond the existing paradigms and propose a novel approach to generate high-quality paraphrases with data of weak supervision. Specifically, we tackle the weakly-supervised paraphrase generation problem by: (1) obtaining abundant weakly-labeled parallel sentences via retrieval-based pseudo paraphrase expansion; and (2) developing a meta-learning framework to progressively select valuable samples for fine-tuning a pre-trained language model BART on the sentential paraphrasing task. We demonstrate that our approach achieves significant improvements over existing unsupervised approaches, and is even comparable in performance with supervised state-of-the-arts.
Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. In this paper, we propose a new method with the goal
of learning a better representation of the style and the content. This method is mainly motivated by the recent success of contrastive learning which has demonstrated its power in unsupervised feature extraction tasks. The idea is to design two contrastive losses with respect to the content and the style by considering two problem characteristics during training. One characteristic is that the target sentence shares the same content with the source sentence, and the second characteristic is that the target sentence shares the same style with the exemplar. These two contrastive losses are incorporated into the general encoder-decoder paradigm. Experiments on two datasets, namely QQP-Pos and ParaNMT, demonstrate the effectiveness of our proposed constrastive losses.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
