Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSearchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
وسيطة مخفية للبحث عن النماذج نهاية إلى نهاية مهام التسلسل المتحلل
361
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
الأساليب نهاية إلى نهاية لمهام التسلسل أصبحت شعبية بشكل متزايد. ومع ذلك بالنسبة لمهام التسلسل المعقدة، مثل ترجمة الكلام، فإن الأنظمة التي تتالي أن العديد من النماذج المدربة على المهام الفرعية قد أظهرت متفوقة، مما يشير إلى أن تكوين النظم المتتالية يبسط التعلم وتمكين قدرات البحث المتطورة. في هذا العمل، نقدم إطارا نهاية إلى نهائي يستغل التركيز لتعلم التمثيلات المخفية القابلة للبحث في المراحل المتوسطة لنموذج التسلسل باستخدام المهام الفرعية المتحللة. يمكن تحسين هذه الوسيط المخفي باستخدام بحث الشعاع لتعزيز الأداء العام ويمكنه أيضا دمج النماذج الخارجية في المراحل المتوسطة للشبكة لإعادة النتيجة أو التكيف باتجاه بيانات خارج المجال. مثيل واحد من الإطار المقترح هو نموذج متعدد اللمعان لترجمة الكلام التي تستخرج الوسطيات المخفية القابلة للبحث عن مهمة فرعية للتعرف على الكلام. يوضح النموذج الفوائد المذكورة أعلاه وتفوق على الحالة السابقة من بين الفن من خلال +6 و +3 بلو على مجموعتي الاختبار من Fisher-Callhome وحوالي +3 و +4 بلو على اللغة الإنجليزية والألمانية والإنجليزية - مجموعات اختبار فرنسية من must-c.
End-to-end approaches for sequence tasks are becoming increasingly popular. Yet for complex sequence tasks, like speech translation, systems that cascade several models trained on sub-tasks have shown to be superior, suggesting that the compositionality of cascaded systems simplifies learning and enables sophisticated search capabilities. In this work, we present an end-to-end framework that exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. These hidden intermediates can be improved using beam search to enhance the overall performance and can also incorporate external models at intermediate stages of the network to re-score or adapt towards out-of-domain data. One instance of the proposed framework is a Multi-Decoder model for speech translation that extracts the searchable hidden intermediates from a speech recognition sub-task. The model demonstrates the aforementioned benefits and outperforms the previous state-of-the-art by around +6 and +3 BLEU on the two test sets of Fisher-CallHome and by around +3 and +4 BLEU on the English-German and English-French test sets of MuST-C.
References used
https://aclanthology.org/
rate research
Read More
Attention-based pre-trained language models such as GPT-2 brought considerable progress to end-to-end dialogue modelling. However, they also present considerable risks for task-oriented dialogue, such as lack of knowledge grounding or diversity. To a
ddress these issues, we introduce modified training objectives for language model finetuning, and we employ massive data augmentation via back-translation to increase the diversity of the training data. We further examine the possibilities of combining data from multiples sources to improve performance on the target dataset. We carefully evaluate our contributions with both human and automatic methods. Our model substantially outperforms the baseline on the MultiWOZ data and shows competitive performance with state of the art in both automatic and human evaluation.
Most previous studies on information status (IS) classification and bridging anaphora recognition assume that the gold mention or syntactic tree information is given (Hou et al., 2013; Roesiger et al., 2018; Hou, 2020; Yu and Poesio, 2020). In this p
aper, we propose an end-to-end neural approach for information status classification. Our approach consists of a mention extraction component and an information status assignment component. During the inference time, our system takes a raw text as the input and generates mentions together with their information status. On the ISNotes corpus (Markert et al., 2012), we show that our information status assignment component achieves new state-of-the-art results on fine-grained IS classification based on gold mentions. Furthermore, our system performs significantly better than other baselines for both mention extraction and fine-grained IS classification in the end-to-end setting. Finally, we apply our system on BASHI (Roesiger, 2018) and SciCorp (Roesiger, 2016) to recognize referential bridging anaphora. We find that our end-to-end system trained on ISNotes achieves competitive results on bridging anaphora recognition compared to the previous state-of-the-art system that relies on syntactic information and is trained on the in-domain datasets (Yu and Poesio, 2020).
Successful conversational search systems can present natural, adaptive and interactive shopping experience for online shopping customers. However, building such systems from scratch faces real word challenges from both imperfect product schema/knowle
dge and lack of training dialog data. In this work we first propose ConvSearch, an end-to-end conversational search system that deeply combines the dialog system with search. It leverages the text profile to retrieve products, which is more robust against imperfect product schema/knowledge compared with using product attributes alone. We then address the lack of data challenges by proposing an utterance transfer approach that generates dialogue utterances by using existing dialog from other domains, and leveraging the search behavior data from e-commerce retailer. With utterance transfer, we introduce a new conversational search dataset for online shopping. Experiments show that our utterance transfer method can significantly improve the availability of training dialogue data without crowd-sourcing, and the conversational search system significantly outperformed the best tested baseline.
For each goal-oriented dialog task of interest, large amounts of data need to be collected for end-to-end learning of a neural dialog system. Collecting that data is a costly and time-consuming process. Instead, we show that we can use only a small a
mount of data, supplemented with data from a related dialog task. Naively learning from related data fails to improve performance as the related data can be inconsistent with the target task. We describe a meta-learning based method that selectively learns from the related dialog task data. Our approach leads to significant accuracy improvements in an example dialog task.
This tutorial surveys the latest technical progress of syntactic parsing and the role of syntax in end-to-end natural language processing (NLP) tasks, in which semantic role labeling (SRL) and machine translation (MT) are the representative NLP tasks
that have always been beneficial from informative syntactic clues since a long time ago, though the advance from end-to-end deep learning models shows new results. In this tutorial, we will first introduce the background and the latest progress of syntactic parsing and SRL/NMT. Then, we will summarize the key evidence about the syntactic impacts over these two concerning tasks, and explore the behind reasons from both computational and linguistic backgrounds.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
