Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userRevisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot
إعادة النظر في إعادة صياغة البداية المستندة إلى المحورية: اللغة ليست المحور الاختياري الوحيد
283
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
الرجوع إلى النصوص التي تنقل نفس المعنى بأشكال تعبير مختلفة. أظهرت الأساليب المستندة إلى Pivot، المعروف أيضا باسم ترجمة الرحلة المستديرة، نتائج واعدة في توليد صياغة عالية الجودة. ومع ذلك، فإن الأساليب القائمة على المحور القائمة تعتمد جميعها على اللغة مثل المحور، حيث تكون النصوص الثنائية الثنائية الموازية على نطاق واسع، مطلوبة موازية عالية الجودة. في هذه الورقة، نستكشف جدوى استخدام التمثيلات الدلالية والنزاهة كحوري من أجل إعادة صياغة نصبها. بشكل ملموس، نتحول جملة إلى مجموعة متنوعة من التمثيلات الدلالية أو النحوية المختلفة (بما في ذلك AMR، UD، وتمثيل الدلالي الكامن)، ثم فك تشفير الجملة من التمثيل الدلالي. نسمح أيضا نهجا يستند إلى إعادة الاحتجاط بضغط عملية خط الأنابيب في إطار نهاية إلى نهاية. نقوم بإجراء تجارب تقارن مناهج مختلفة مع أنواع مختلفة من المحاور. تظهر النتائج التجريبية أن اتخاذ AMR كما يمكن للمحافظة الحصول على صياغة مع جودة أفضل من تناول اللغة كمحور. يمكن للإطار الطرفي النهائي أن يقلل من التحول الدلالي عند استخدام اللغة كحوري. إضافة إلى جانب ذلك، يمكن أن تولد العديد من الطرق القائمة على المحور غير المركبات أن تولد صياغة مع جودة مماثلة كنموذج التسلسل الإشراف على التسلسل، والذي يشير إلى أن البيانات الموازية للصيانة قد لا تكون ضرورية لإعادة صياغة عناصر التوليد.
Paraphrases refer to texts that convey the same meaning with different expression forms. Pivot-based methods, also known as the round-trip translation, have shown promising results in generating high-quality paraphrases. However, existing pivot-based methods all rely on language as the pivot, where large-scale, high-quality parallel bilingual texts are required. In this paper, we explore the feasibility of using semantic and syntactic representations as the pivot for paraphrase generation. Concretely, we transform a sentence into a variety of different semantic or syntactic representations (including AMR, UD, and latent semantic representation), and then decode the sentence back from the semantic representations. We further explore a pretraining-based approach to compress the pipeline process into an end-to-end framework. We conduct experiments comparing different approaches with different kinds of pivots. Experimental results show that taking AMR as pivot can obtain paraphrases with better quality than taking language as the pivot. The end-to-end framework can reduce semantic shift when language is used as the pivot. Besides, several unsupervised pivot-based methods can generate paraphrases with similar quality as the supervised sequence-to-sequence model, which indicates that parallel data of paraphrases may not be necessary for paraphrase generation.
References used
https://aclanthology.org/
rate research
Read More
Pretrained language models have served as the backbone for many state-of-the-art NLP results. These models are large and expensive to train. Recent work suggests that continued pretraining on task-specific data is worth the effort as pretraining lead
s to improved performance on downstream tasks. We explore alternatives to full-scale task-specific pretraining of language models through the use of adapter modules, a parameter-efficient approach to transfer learning. We find that adapter-based pretraining is able to achieve comparable results to task-specific pretraining while using a fraction of the overall trainable parameters. We further explore direct use of adapters without pretraining and find that the direct fine-tuning performs mostly on par with pretrained adapter models, contradicting previously proposed benefits of continual pretraining in full pretraining fine-tuning strategies. Lastly, we perform an ablation study on task-adaptive pretraining to investigate how different hyperparameter settings can change the effectiveness of the pretraining.
Paraphrase generation is an important task in natural language processing. Previous works focus on sentence-level paraphrase generation, while ignoring document-level paraphrase generation, which is a more challenging and valuable task. In this paper
, we explore the task of document-level paraphrase generation for the first time and focus on the inter-sentence diversity by considering sentence rewriting and reordering. We propose CoRPG (Coherence Relationship guided Paraphrase Generation), which leverages graph GRU to encode the coherence relationship graph and get the coherence-aware representation for each sentence, which can be used for re-arranging the multiple (possibly modified) input sentences. We create a pseudo document-level paraphrase dataset for training CoRPG. Automatic evaluation results show CoRPG outperforms several strong baseline models on the BERTScore and diversity scores. Human evaluation also shows our model can generate document paraphrase with more diversity and semantic preservation.
This work explores the capacities of character-based Neural Machine Translation to translate noisy User-Generated Content (UGC) with a strong focus on exploring the limits of such approaches to handle productive UGC phenomena, which almost by definit
ion, cannot be seen at training time. Within a strict zero-shot scenario, we first study the detrimental impact on translation performance of various user-generated content phenomena on a small annotated dataset we developed and then show that such models are indeed incapable of handling unknown letters, which leads to catastrophic translation failure once such characters are encountered. We further confirm this behavior with a simple, yet insightful, copy task experiment and highlight the importance of reducing the vocabulary size hyper-parameter to increase the robustness of character-based models for machine translation.
Recent progress in language modeling has been driven not only by advances in neural architectures, but also through hardware and optimization improvements. In this paper, we revisit the neural probabilistic language model (NPLM) of Bengio et al. (200
3), which simply concatenates word embeddings within a fixed window and passes the result through a feed-forward network to predict the next word. When scaled up to modern hardware, this model (despite its many limitations) performs much better than expected on word-level language model benchmarks. Our analysis reveals that the NPLM achieves lower perplexity than a baseline Transformer with short input contexts but struggles to handle long-term dependencies. Inspired by this result, we modify the Transformer by replacing its first self-attention layer with the NPLM's local concatenation layer, which results in small but consistent perplexity decreases across three word-level language modeling datasets.
This paper focuses on paraphrase generation,which is a widely studied natural language generation task in NLP. With the development of neural models, paraphrase generation research has exhibited a gradual shift to neural methods in the recent years.
This has provided architectures for contextualized representation of an input text and generating fluent, diverseand human-like paraphrases. This paper surveys various approaches to paraphrase generation with a main focus on neural methods.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
