Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userIncorporating speaker embedding and post-filter network for improving speaker similarity of personalized speech synthesis system
دمج متكلم التضمين وشبكة ما بعد المرشح لتحسين تشابه المتكلم لنظام توليف الكلام المخصص
370
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في السنوات الأخيرة، يمكن لنظام توليف الكلام إنشاء خطاب بجودة الكلام العالية. ومع ذلك، لا يزال نظام النص إلى كلام متعدد الكلام (TTS) يتطلب كمية كبيرة من بيانات الكلام لكل مكبر صوت مستهدف. في هذه الدراسة، نود إنشاء نظام TTS متعدد المتكلم من خلال دمج وحدات فرعية في نظام توليف الكلام المصنوع من الشبكة العصبية المصطنعة لتخفيف هذه المشكلة. تتمثل الوحدة الأولى في إضافة مكبر صوت إلى وحدة الترميز لتوليد الكلام في حين أن كمية كبيرة من بيانات الكلام من السماعة المستهدفة ليست ضرورية. بالنسبة لطريقة تضمين المتكلم، في دراستنا، يتم مقارنة طريقتان رئيسيتان لضمان المتكلم، وهي تضمين مكبر الصوت التضمين وإدماج تحويل الصوت، بتحديد ما هو مناسب لنظام TTS الشخصي الخاص بنا. ثانيا، استبدلنا الوحدة النمطية الإضافية التقليدية، التي تم تبنيها لتعزيز تسلسل طيف الإخراج، لزيادة تحسين جودة خطاب الكلام الذي تم إنشاؤه. هنا، يتم استخدام شبكة ما بعد المرشح. أخيرا، أظهرت نتائج التجربة أن تضمين المتكلم مفيد من خلال إضافةه إلى وحدة ترميز ونطق الكلام الناتج ينظر بالفعل إلى السماعة المستهدفة. أيضا، شبكة ما بعد التصفية ليست فقط تحسين جودة الكلام وتعزز أيضا تشابه المتكلم من كلام الكلام الناتج. يمكن لنظام TTS المصمم أن يولد كلام الكلام للمتكلم المستهدف في أقل من 2 ثانية. في المستقبل، نود مزيد من التحقيق في مكافحة قابلية التحكم في معدل التحدث أو حالة المشاعر المتصورة للكلمة التي تم إنشاؤها.
In recent years, speech synthesis system can generate speech with high speech quality. However, multi-speaker text-to-speech (TTS) system still require large amount of speech data for each target speaker. In this study, we would like to construct a multi-speaker TTS system by incorporating two sub modules into artificial neural network-based speech synthesis system to alleviate this problem. First module is to add speaker embedding into encoding module for generating speech while a large amount of the speech data from target speaker is not necessary. For speaker embedding method, in our study, two main speaker embedding methods, namely speaker verification embedding and voice conversion embedding, are compared to deciding which one is suitable for our personalized TTS system. Second, we substituted the conventional post-net module, which is adopted to enhance the output spectrum sequence, to further improving the speech quality of the generated speech utterance. Here, a post-filter network is used. Finally, experiment results showed that the speaker embedding is useful by adding it into encoding module and the resultant speech utterance indeed perceived as the target speaker. Also, the post-filter network not only improving the speech quality and also enhancing the speaker similarity of the generated speech utterances. The constructed TTS system can generate a speech utterance of the target speaker in fewer than 2 seconds. In the future, we would like to further investigate the controllability of the speaking rate or perceived emotion state of the generated speech.
References used
https://aclanthology.org/
rate research
Read More
In this paper, we focus on improving the quality of the summary generated by neural abstractive dialogue summarization systems. Even though pre-trained language models generate well-constructed and promising results, it is still challenging to summar
ize the conversation of multiple participants since the summary should include a description of the overall situation and the actions of each speaker. This paper proposes self-supervised strategies for speaker-focused post-correction in abstractive dialogue summarization. Specifically, our model first discriminates which type of speaker correction is required in a draft summary and then generates a revised summary according to the required type. Experimental results show that our proposed method adequately corrects the draft summaries, and the revised summaries are significantly improved in both quantitative and qualitative evaluations.
For children, the system trained on a large corpus of adult speakers performed worse than a system trained on a much smaller corpus of children's speech. This is due to the acoustic mismatch between training and testing data. To capture more acoustic
variability we trained a shared system with mixed data from adults and children. The shared system yields the best EER for children with no degradation for adults. Thus, the single system trained with mixed data is applicable for speaker verification for both adults and children.
Product quantization (PQ) is a widely used technique for ad-hoc retrieval. Recent studies propose supervised PQ, where the embedding and quantization models can be jointly trained with supervised learning. However, there is a lack of appropriate form
ulation of the joint training objective; thus, the improvements over previous non-supervised baselines are limited in reality. In this work, we propose the Matching-oriented Product Quantization (MoPQ), where a novel objective Multinoulli Contrastive Loss (MCL) is formulated. With the minimization of MCL, we are able to maximize the matching probability of query and ground-truth key, which contributes to the optimal retrieval accuracy. Given that the exact computation of MCL is intractable due to the demand of vast contrastive samples, we further propose the Differentiable Cross-device Sampling (DCS), which significantly augments the contrastive samples for precise approximation of MCL. We conduct extensive experimental studies on four real-world datasets, whose results verify the effectiveness of MoPQ. The code is available at https://github.com/microsoft/MoPQ.
Dialogue Act (DA) classification is the task of classifying utterances with respect to the function they serve in a dialogue. Existing approaches to DA classification model utterances without incorporating the turn changes among speakers throughout t
he dialogue, therefore treating it no different than non-interactive written text. In this paper, we propose to integrate the turn changes in conversations among speakers when modeling DAs. Specifically, we learn conversation-invariant speaker turn embeddings to represent the speaker turns in a conversation; the learned speaker turn embeddings are then merged with the utterance embeddings for the downstream task of DA classification. With this simple yet effective mechanism, our model is able to capture the semantics from the dialogue content while accounting for different speaker turns in a conversation. Validation on three benchmark public datasets demonstrates superior performance of our model.
Abstractive summarization quality had large improvements since recent language pretraining techniques. However, currently there is a lack of datasets for the growing needs of conversation summarization applications. Thus we collected ForumSum, a dive
rse and high-quality conversation summarization dataset with human written summaries. The conversations in ForumSum dataset are collected from a wide variety of internet forums. To make the dataset easily expandable, we also release the process of dataset creation. Our experiments show that models trained on ForumSum have better zero-shot and few-shot transferability to other datasets than the existing large chat summarization dataset SAMSum. We also show that using a conversational corpus for pre-training improves the quality of the chat summarization model.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
