Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSource and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation
المصدر والهدف تقطير المعرفة ثنائية الاتجاه لترجمة خطاب نهاية إلى نهاية
264
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تتمثل النهج التقليدي في تحسين أداء نماذج ترجمة الكلام في النهاية (E2E-St) في الاستفادة من النسخ المصدر عبر التدريب المسبق والتدريب المشترك مع التعرف على الكلام التلقائي (ASR) ومهام الترجمة الآلية العصبية (NMT). ومع ذلك، نظرا لأن طرائق الإدخال مختلفة، فمن الصعب الاستفادة من نص لغة المصدر بنجاح. في هذا العمل، نركز على تقطير المعرفة على مستوى التسلسل (SEQKD) من نماذج NMT الخارجية القائمة على النصوص. للاستفادة من الإمكانات الكاملة لمعلومات اللغة المصدر، نقترحنا على الوراء SEQKD، SEQKD من نموذج NMT للخلف إلى المصدر. تحقيقا لهذه الغاية، نقوم بتدريب نموذج ثنائي اللغة E2E-St للتنبؤ بالصايات بمثابة مهمة مساعدة مع وحدة فك ترميز واحدة. يتم إنشاء الصيغة من الترجمات في Bitex عبر الترجمة مرة أخرى. ونحن نقترح مزيدا من SEQKD ثنائي الاتجاه حيث يتم دمج SEQKD من نماذج NMT إلى الأمام والخلف. تظهر التقييمات التجريبية على كل من النماذج التلقائية وغير اللاحنة التلقائية أن Seqkd في كل اتجاه يحسن باستمرار أداء الترجمة، والفعالية مكملة بغض النظر عن القدرات النموذجية.
A conventional approach to improving the performance of end-to-end speech translation (E2E-ST) models is to leverage the source transcription via pre-training and joint training with automatic speech recognition (ASR) and neural machine translation (NMT) tasks. However, since the input modalities are different, it is difficult to leverage source language text successfully. In this work, we focus on sequence-level knowledge distillation (SeqKD) from external text-based NMT models. To leverage the full potential of the source language information, we propose backward SeqKD, SeqKD from a target-to-source backward NMT model. To this end, we train a bilingual E2E-ST model to predict paraphrased transcriptions as an auxiliary task with a single decoder. The paraphrases are generated from the translations in bitext via back-translation. We further propose bidirectional SeqKD in which SeqKD from both forward and backward NMT models is combined. Experimental evaluations on both autoregressive and non-autoregressive models show that SeqKD in each direction consistently improves the translation performance, and the effectiveness is complementary regardless of the model capacity.
References used
https://aclanthology.org/
rate research
Read More
This paper describes the submission of the NiuTrans end-to-end speech translation system for the IWSLT 2021 offline task, which translates from the English audio to German text directly without intermediate transcription. We use the Transformer-based
model architecture and enhance it by Conformer, relative position encoding, and stacked acoustic and textual encoding. To augment the training data, the English transcriptions are translated to German translations. Finally, we employ ensemble decoding to integrate the predictions from several models trained with the different datasets. Combining these techniques, we achieve 33.84 BLEU points on the MuST-C En-De test set, which shows the enormous potential of the end-to-end model.
Recent studies argue that knowledge distillation is promising for speech translation (ST) using end-to-end models. In this work, we investigate the effect of knowledge distillation with a cascade ST using automatic speech recognition (ASR) and machin
e translation (MT) models. We distill knowledge from a teacher model based on human transcripts to a student model based on erroneous transcriptions. Our experimental results demonstrated that knowledge distillation is beneficial for a cascade ST. Further investigation that combined knowledge distillation and fine-tuning revealed that the combination consistently improved two language pairs: English-Italian and Spanish-English.
Recently, end-to-end (E2E) trained models for question answering over knowledge graphs (KGQA) have delivered promising results using only a weakly supervised dataset. However, these models are trained and evaluated in a setting where hand-annotated q
uestion entities are supplied to the model, leaving the important and non-trivial task of entity resolution (ER) outside the scope of E2E learning. In this work, we extend the boundaries of E2E learning for KGQA to include the training of an ER component. Our model only needs the question text and the answer entities to train, and delivers a stand-alone QA model that does not require an additional ER component to be supplied during runtime. Our approach is fully differentiable, thanks to its reliance on a recent method for building differentiable KGs (Cohen et al., 2020). We evaluate our E2E trained model on two public datasets and show that it comes close to baseline models that use hand-annotated entities.
Most previous studies on information status (IS) classification and bridging anaphora recognition assume that the gold mention or syntactic tree information is given (Hou et al., 2013; Roesiger et al., 2018; Hou, 2020; Yu and Poesio, 2020). In this p
aper, we propose an end-to-end neural approach for information status classification. Our approach consists of a mention extraction component and an information status assignment component. During the inference time, our system takes a raw text as the input and generates mentions together with their information status. On the ISNotes corpus (Markert et al., 2012), we show that our information status assignment component achieves new state-of-the-art results on fine-grained IS classification based on gold mentions. Furthermore, our system performs significantly better than other baselines for both mention extraction and fine-grained IS classification in the end-to-end setting. Finally, we apply our system on BASHI (Roesiger, 2018) and SciCorp (Roesiger, 2016) to recognize referential bridging anaphora. We find that our end-to-end system trained on ISNotes achieves competitive results on bridging anaphora recognition compared to the previous state-of-the-art system that relies on syntactic information and is trained on the in-domain datasets (Yu and Poesio, 2020).
This tutorial surveys the latest technical progress of syntactic parsing and the role of syntax in end-to-end natural language processing (NLP) tasks, in which semantic role labeling (SRL) and machine translation (MT) are the representative NLP tasks
that have always been beneficial from informative syntactic clues since a long time ago, though the advance from end-to-end deep learning models shows new results. In this tutorial, we will first introduce the background and the latest progress of syntactic parsing and SRL/NMT. Then, we will summarize the key evidence about the syntactic impacts over these two concerning tasks, and explore the behind reasons from both computational and linguistic backgrounds.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
