Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSimultaneous Speech Translation for Live Subtitling: from Delay to Display
ترجمة خطاب متزامنة للترجمة الحية: من التأخير لعرضها
619
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
مع زيادة الصوت في مجال الاتصالات، فإن الحاجة إلى ترجمات حية في الأحداث متعددة اللغات هي أكثر أهمية من أي وقت مضى. في محاولة لأتمتة العملية، نهدف إلى استكشاف جدوى ترجمة الكلام في وقت واحد (Simulst) للترجمة الحية. ومع ذلك، فإن معدل توليد Word-For-Word من أنظمة Simulst غير مثلى لعرض الترجمات بطريقة مفهومة وقابلة للقراءة. في هذا العمل، نقوم بتكييف Simulst Systems للتنبؤ برصيد الترجمة إلى جانب الترجمة. بعد ذلك، نقترح وضع العرض الذي يستغل بنية الاستراحة المتوقعة عن طريق تقديم الترجمات في خطوط التمرير. قارنا وضعنا المقترح مع شاشة 1) Word-for-word و 2) في كتل، من حيث سرعة القراءة والتأخير. تظهر التجارب على ثلاث أزواج اللغة (en → IT، DE، FR) أن خطوط التمرير هي الوضع الوحيد الذي يحقق سرعة قراءة مقبولة مع الحفاظ على التأخير بالقرب من عتبة 4 ثانية. نجرب أن الترجمة الفورية للترجمات الحية القابلة للقراءة لا تزال تواجه تحديات، والثيقة هي ذات جودة ترجمة ضعيفة، واقتراح توجيهات بحث في المستقبل.
With the increased audiovisualisation of communication, the need for live subtitles in multilingual events is more relevant than ever. In an attempt to automatise the process, we aim at exploring the feasibility of simultaneous speech translation (SimulST) for live subtitling. However, the word-for-word rate of generation of SimulST systems is not optimal for displaying the subtitles in a comprehensible and readable way. In this work, we adapt SimulST systems to predict subtitle breaks along with the translation. We then propose a display mode that exploits the predicted break structure by presenting the subtitles in scrolling lines. We compare our proposed mode with a display 1) word-for-word and 2) in blocks, in terms of reading speed and delay. Experiments on three language pairs (en→it, de, fr) show that scrolling lines is the only mode achieving an acceptable reading speed while keeping delay close to a 4-second threshold. We argue that simultaneous translation for readable live subtitles still faces challenges, the main one being poor translation quality, and propose directions for steering future research.
References used
https://aclanthology.org/
rate research
Read More
Traditional translation systems trained on written documents perform well for text-based translation but not as well for speech-based applications. We aim to adapt translation models to speech by introducing actual lexical errors from ASR and segment
ation errors from automatic punctuation into our translation training data. We introduce an inverted projection approach that projects automatically detected system segments onto human transcripts and then re-segments the gold translations to align with the projected human transcripts. We demonstrate that this overcomes the train-test mismatch present in other training approaches. The new projection approach achieves gains of over 1 BLEU point over a baseline that is exposed to the human transcripts and segmentations, and these gains hold for both IWSLT data and YouTube data.
In recent years, automatic speech-to-speech and speech-to-text translation has gained momentum thanks to advances in artificial intelligence, especially in the domains of speech recognition and machine translation. The quality of such applications is
commonly tested with automatic metrics, such as BLEU, primarily with the goal of assessing improvements of releases or in the context of evaluation campaigns. However, little is known about how the output of such systems is perceived by end users or how they compare to human performances in similar communicative tasks. In this paper, we present the results of an experiment aimed at evaluating the quality of a real-time speech translation engine by comparing it to the performance of professional simultaneous interpreters. To do so, we adopt a framework developed for the assessment of human interpreters and use it to perform a manual evaluation on both human and machine performances. In our sample, we found better performance for the human interpreters in terms of intelligibility, while the machine performs slightly better in terms of informativeness. The limitations of the study and the possible enhancements of the chosen framework are discussed. Despite its intrinsic limitations, the use of this framework represents a first step towards a user-centric and communication-oriented methodology for evaluating real-time automatic speech translation.
This paper describes Maastricht University's participation in the IWSLT 2021 multilingual speech translation track. The task in this track is to build multilingual speech translation systems in supervised and zero-shot directions. Our primary system
is an end-to-end model that performs both speech transcription and translation. We observe that the joint training for the two tasks is complementary especially when the speech translation data is scarce. On the source and target side, we use data augmentation and pseudo-labels respectively to improve the performance of our systems. We also introduce an ensembling technique that consistently improves the quality of transcriptions and translations. The experiments show that the end-to-end system is competitive with its cascaded counterpart especially in zero-shot conditions.
This paper describes the offline and simultaneous speech translation systems developed at AppTek for IWSLT 2021. Our offline ST submission includes the direct end-to-end system and the so-called posterior tight integrated model, which is akin to the
cascade system but is trained in an end-to-end fashion, where all the cascaded modules are end-to-end models themselves. For simultaneous ST, we combine hybrid automatic speech recognition with a machine translation approach whose translation policy decisions are learned from statistical word alignments. Compared to last year, we improve general quality and provide a wider range of quality/latency trade-offs, both due to a data augmentation method making the MT model robust to varying chunk sizes. Finally, we present a method for ASR output segmentation into sentences that introduces a minimal additional delay.
We propose a generative framework for simultaneous machine translation. Conventional approaches use a fixed number of source words to translate or learn dynamic policies for the number of source words by reinforcement learning. Here we formulate simu
ltaneous translation as a structural sequence-to-sequence learning problem. A latent variable is introduced to model read or translate actions at every time step, which is then integrated out to consider all the possible translation policies. A re-parameterised Poisson prior is used to regularise the policies which allows the model to explicitly balance translation quality and latency. The experiments demonstrate the effectiveness and robustness of the generative framework, which achieves the best BLEU scores given different average translation latencies on benchmark datasets.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
