Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTowards Code-Mixed Hinglish Dialogue Generation
نحو توليد الحوار الهندي المختلط
247
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تلعب اللغة المختلطة من التعليمات البرمجية دورا حاسما في الاتصالات في المجتمعات متعددة اللغات. على الرغم من أن النمو الأخير لمستخدمي الويب قد عززوا إلى حد كبير استخدام مثل هذه اللغات المختلطة، فإن الجيل الحالي لأنظمة الحوار مونولجة في المقام الأول. هذه الزيادة في استخدام اللغة المختلطة من التعليمات البرمجية قد دفعت أنظمة الحوار بلغة مماثلة. نقدم عملنا في توليد الحوار المختلط من التعليمات البرمجية، وهي مهمة غير مستكشفة في اللغات المختلطة من التعليمات البرمجية، وتوليد الكلام في اللغة المختلطة من التعليمات البرمجية بدلا من لغة واحدة في كثير من الأحيان الإنجليزية فقط. نقدم لجنة اصطناعية جديدة في مزيج التعليمات البرمجية للحوائط، CM-DAYAYDIALOG، عن طريق تحويل كوربوس حوار موجود باللغة الإنجليزية فقط إلى Corpus مختلطة باللغة الهندية. بعد ذلك اقترحنا نهجا أساسيا حيث نظهر فعالية استخدام MBART مثل محولات تسلسل تسلسل متعدد اللغات لتوليد الحوار المختلط. يمكن لأفضل طرازات الحوار الأداء لدينا إجراء محادثات متماسكة في اللغة المختلطة الهندية - الإنجليزية كما تم تقييمها بواسطة المقاييس البشرية والآلية التي تحدد معايير جديدة لمهمة توليد الحوار المختلط من التعليمات البرمجية.
Code-mixed language plays a crucial role in communication in multilingual societies. Though the recent growth of web users has greatly boosted the use of such mixed languages, the current generation of dialog systems is primarily monolingual. This increase in usage of code-mixed language has prompted dialog systems in a similar language. We present our work in Code-Mixed Dialog Generation, an unexplored task in code-mixed languages, generating utterances in code-mixed language rather than a single language that is more often just English. We present a new synthetic corpus in code-mix for dialogs, CM-DailyDialog, by converting an existing English-only dialog corpus to a mixed Hindi-English corpus. We then propose a baseline approach where we show the effectiveness of using mBART like multilingual sequence-to-sequence transformers for code-mixed dialog generation. Our best performing dialog models can conduct coherent conversations in Hindi-English mixed language as evaluated by human and automatic metrics setting new benchmarks for the Code-Mixed Dialog Generation task.
References used
https://aclanthology.org/
rate research
Read More
In this shared task, we seek the participating teams to investigate the factors influencing the quality of the code-mixed text generation systems. We synthetically generate code-mixed Hinglish sentences using two distinct approaches and employ human
annotators to rate the generation quality. We propose two subtasks, quality rating prediction and annotators' disagreement prediction of the synthetic Hinglish dataset. The proposed subtasks will put forward the reasoning and explanation of the factors influencing the quality and human perception of the code-mixed text.
The increasing use of social media sites in countries like India has given rise to large volumes of code-mixed data. Sentiment analysis of this data can provide integral insights into people's perspectives and opinions. Code-mixed data is often noisy
in nature due to multiple spellings for the same word, lack of definite order of words in a sentence, and random abbreviations. Thus, working with code-mixed data is more challenging than monolingual data. Interpreting a model's predictions allows us to determine the robustness of the model against different forms of noise. In this paper, we propose a methodology to integrate explainable approaches into code-mixed sentiment analysis. By interpreting the predictions of sentiment analysis models we evaluate how well the model is able to adapt to the implicit noises present in code-mixed data.
Text generation is a highly active area of research in the computational linguistic community. The evaluation of the generated text is a challenging task and multiple theories and metrics have been proposed over the years. Unfortunately, text generat
ion and evaluation are relatively understudied due to the scarcity of high-quality resources in code-mixed languages where the words and phrases from multiple languages are mixed in a single utterance of text and speech. To address this challenge, we present a corpus (HinGE) for a widely popular code-mixed language Hinglish (code-mixing of Hindi and English languages). HinGE has Hinglish sentences generated by humans as well as two rule-based algorithms corresponding to the parallel Hindi-English sentences. In addition, we demonstrate the in- efficacy of widely-used evaluation metrics on the code-mixed data. The HinGE dataset will facilitate the progress of natural language generation research in code-mixed languages.
A major challenge in analysing social me-dia data belonging to languages that use non-English script is its code-mixed nature. Recentresearch has presented state-of-the-art contex-tual embedding models (both monolingual s.a.BERT and multilingual s.a.
XLM-R) as apromising approach. In this paper, we showthat the performance of such embedding mod-els depends on multiple factors, such as thelevel of code-mixing in the dataset, and thesize of the training dataset. We empiricallyshow that a newly introduced Capsule+biGRUclassifier could outperform a classifier built onthe English-BERT as well as XLM-R just witha training dataset of about 6500 samples forthe Sinhala-English code-mixed data.
Pre-trained language-vision models have shown remarkable performance on the visual question answering (VQA) task. However, most pre-trained models are trained by only considering monolingual learning, especially the resource-rich language like Englis
h. Training such models for multilingual setups demand high computing resources and multilingual language-vision dataset which hinders their application in practice. To alleviate these challenges, we propose a knowledge distillation approach to extend an English language-vision model (teacher) into an equally effective multilingual and code-mixed model (student). Unlike the existing knowledge distillation methods, which only use the output from the last layer of the teacher network for distillation, our student model learns and imitates the teacher from multiple intermediate layers (language and vision encoders) with appropriately designed distillation objectives for incremental knowledge extraction. We also create the large-scale multilingual and code-mixed VQA dataset in eleven different language setups considering the multiple Indian and European languages. Experimental results and in-depth analysis show the effectiveness of the proposed VQA model over the pre-trained language-vision models on eleven diverse language setups.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
