Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDealing with training and test segmentation mismatch: FBK@IWSLT2021
التعامل مع عدم تطابق تجزئة التدريب والاختبار: FBK @ IWSLT2021
161
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
توضح هذه الورقة تقديم نظام FBK إلى مهمة ترجمة الكلام IWSLT 2021. شاركنا نموذجا مباشرا، وهو عبارة عن بنية قائمة على المحولات المدربة لتترجم بيانات صوت الكلام باللغة الإنجليزية إلى نصوص ألمانية. يتميز خط أنابيب التدريب بتقطير المعرفة وإجراءات ضبط ناعم من خطوتين. يتم تنفيذ كل من تقطير المعرفة والخطوة الأولى الدقيقة على البيانات الحقيقية والاصطناعية مجزأة يدويا، والذي يتم إنشاؤه في هذا الأخير بنظام MT المدرب على الشركة المتاحة. بشكل مختلف، يتم تنفيذ خطوة التشغيل الدقيقة الثانية في تجزئة عشوائية ل DataSet MUST-C V2 EN-DE. هدفها الرئيسي هو تقليل قطرات الأداء التي تحدث عندما يتم تقييم نموذج الترجمة الكلام على البيانات المجزأة يدويا (I.E. لنفس الغرض، يتم تطبيق إجراء تجزئة هجينة مخصص تمثل كلا المحتوى الصوتي (توقف) وطول القطاعات المنتجة على بيانات الاختبار قبل تمريرها إلى النظام. في وقت الاستنتاج، قارننا هذا الإجراء مع طريقة تجزئة خط الأساس استنادا إلى اكتشاف نشاط الصوت (VAD). تشير نتائجنا إلى فعالية النهج الهجين المقترح، الذي أظهره تخفيض الفجوة مع تجزئة يدوية من نقاط بلو 8.3 إلى 1.4.
This paper describes FBK's system submission to the IWSLT 2021 Offline Speech Translation task. We participated with a direct model, which is a Transformer-based architecture trained to translate English speech audio data into German texts. The training pipeline is characterized by knowledge distillation and a two-step fine-tuning procedure. Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora. Differently, the second fine-tuning step is carried out on a random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce the performance drops occurring when a speech translation model trained on manually segmented data (i.e. an ideal, sentence-like segmentation) is evaluated on automatically segmented audio (i.e. actual, more realistic testing conditions). For the same purpose, a custom hybrid segmentation procedure that accounts for both audio content (pauses) and for the length of the produced segments is applied to the test data before passing them to the system. At inference time, we compared this procedure with a baseline segmentation method based on Voice Activity Detection (VAD). Our results indicate the effectiveness of the proposed hybrid approach, shown by a reduction of the gap with manual segmentation from 8.3 to 1.4 BLEU points.
References used
https://aclanthology.org/
rate research
Read More
Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS). However, PTMs used in previous works usually adopt language modeling as pre-training tasks, lacking task-specific prior segmentation knowledge an
d ignoring the discrepancy between pre-training tasks and downstream CWS tasks. In this paper, we propose a CWS-specific pre-trained model MetaSeg, which employs a unified architecture and incorporates meta learning algorithm into a multi-criteria pre-training task. Empirical results show that MetaSeg could utilize common prior segmentation knowledge from different existing criteria and alleviate the discrepancy between pre-trained models and downstream CWS tasks. Besides, MetaSeg can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings.
In quality estimation (QE), the quality of translation can be predicted by referencing the source sentence and the machine translation (MT) output without access to the reference sentence. However, there exists a paradox in that constructing a datase
t for creating a QE model requires non-trivial human labor and time, and it may even requires additional effort compared to the cost of constructing a parallel corpus. In this study, to address this paradox and utilize the various applications of QE, even in low-resource languages (LRLs), we propose a method for automatically constructing a pseudo-QE dataset without using human labor. We perform a comparative analysis on the pseudo-QE dataset using multilingual pre-trained language models. As we generate the pseudo dataset, we conduct experiments using various external machine translators as test sets to verify the accuracy of the results objectively. Also, the experimental results show that multilingual BART demonstrates the best performance, and we confirm the applicability of QE in LRLs using pseudo-QE dataset construction methods.
We present new state-of-the-art benchmarks for paraphrase detection on all six languages in the Opusparcus sentential paraphrase corpus: English, Finnish, French, German, Russian, and Swedish. We reach these baselines by fine-tuning BERT. The best re
sults are achieved on smaller and cleaner subsets of the training sets than was observed in previous research. Additionally, we study a translation-based approach that is competitive for the languages with more limited and noisier training data.
Passage retrieval and ranking is a key task in open-domain question answering and information retrieval. Current effective approaches mostly rely on pre-trained deep language model-based retrievers and rankers. These methods have been shown to effect
ively model the semantic matching between queries and passages, also in presence of keyword mismatch, i.e. passages that are relevant to a query but do not contain important query keywords. In this paper we consider the Dense Retriever (DR), a passage retrieval method, and the BERT re-ranker, a popular passage re-ranking method. In this context, we formally investigate how these models respond and adapt to a specific type of keyword mismatch -- that caused by keyword typos occurring in queries. Through empirical investigation, we find that typos can lead to a significant drop in retrieval and ranking effectiveness. We then propose a simple typos-aware training framework for DR and BERT re-ranker to address this issue. Our experimental results on the MS MARCO passage ranking dataset show that, with our proposed typos-aware training, DR and BERT re-ranker can become robust to typos in queries, resulting in significantly improved effectiveness compared to models trained without appropriately accounting for typos.
This paper contains the description for the submission of Karlsruhe Institute of Technology (KIT) for the multilingual TEDx translation task in the IWSLT 2021 evaluation campaign. Our main approach is to develop both cascade and end-to-end systems an
d eventually combine them together to achieve the best possible results for this extremely low-resource setting. The report also confirms certain consistent architectural improvement added to the Transformer architecture, for all tasks: translation, transcription and speech translation.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
