Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userMultimodal End-to-End Sparse Model for Emotion Recognition
نموذج متفرق متعدد الإنهاء للتعرف على العاطفة
407
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يعمل المصنف الموجود في مهام الحوسبة العاطفية متعددة الوسائط، مثل التعرف على العاطفة والتعرف على الشخصية، عموما خط أنابيب ذات مرحلتين من خلال أول استخراج تمثيلات ميزة لكل طريقة واحدة مع الخوارزميات المصنوعة يدويا، ثم أداء التعلم المنتهي مع الميزات المستخرجة. ومع ذلك، يتم إصلاح الميزات المستخرجة ولا يمكن ضبطها بشكل جيد على المهام المستهدفة المختلفة، والعثور على ميزة خوارزميات الاستخراج يدويا لا تعميم أو مقياس جيدا لمهام مختلفة، والتي يمكن أن تؤدي إلى الأداء دون الأمثل. في هذه الورقة، نقوم بتطوير نموذج طرف بالكامل يربط المرحلتين وتحسينها بشكل مشترك. بالإضافة إلى ذلك، نقوم بإعادة هيكلة مجموعات البيانات الحالية لتمكين التدريب الكامل للنهاية. علاوة على ذلك، لتقليل النفقات الحاسوبية النماذج المحسوبة بالنماذج الطرفية إلى النهاية، نقدم آلية اهتمامية متناثرة عبر مشروط لاستخراج الميزة. تظهر النتائج التجريبية أن طرازنا الناتج الكامل ينفج بشكل كبير يتفوق بشكل كبير النماذج الحالية للحالة القائمة على خط أنابيب الطورين. علاوة على ذلك، من خلال إضافة اهتمام متناثر عبر الوسائط، يمكن لنموذجنا الحفاظ على الأداء مع حوالي نصف حساب أقل في جزء استخراج الميزة من النموذج.
Existing works in multimodal affective computing tasks, such as emotion recognition and personality recognition, generally adopt a two-phase pipeline by first extracting feature representations for each single modality with hand crafted algorithms, and then performing end-to-end learning with extracted features. However, the extracted features are fixed and cannot be further fine-tuned on different target tasks, and manually finding feature extracting algorithms does not generalize or scale well to different tasks, which can lead to sub-optimal performance. In this paper, we develop a fully end-to-end model that connects the two phases and optimizes them jointly. In addition, we restructure the current datasets to enable the fully end-to-end training. Furthermore, to reduce the computational overhead brought by the end-to-end model, we introduce a sparse cross-modal attention mechanism for the feature extraction. Experimental results show that our fully end-to-end model significantly surpasses the current state-of-the-art models based on the two-phase pipeline. Moreover, by adding the sparse cross-modal attention, our model can maintain the performance with around half less computation in the feature extraction part of the model.
References used
https://aclanthology.org/
rate research
Read More
Due to the popularity of intelligent dialogue assistant services, speech emotion recognition has become more and more important. In the communication between humans and machines, emotion recognition and emotion analysis can enhance the interaction be
tween machines and humans. This study uses the CNN+LSTM model to implement speech emotion recognition (SER) processing and prediction. From the experimental results, it is known that using the CNN+LSTM model achieves better performance than using the traditional NN model.
Emotion recognition in conversation has received considerable attention recently because of its practical industrial applications. Existing methods tend to overlook the immediate mutual interaction between different speakers in the speaker-utterance
level, or apply single speaker-agnostic RNN for utterances from different speakers. We propose COIN, a conversational interactive model to mitigate this problem by applying state mutual interaction within history contexts. In addition, we introduce a stacked global interaction module to capture the contextual and inter-dependency representation in a hierarchical manner. To improve the robustness and generalization during training, we generate adversarial examples by applying the minor perturbations on multimodal feature inputs, unveiling the benefits of adversarial examples for emotion detection. The proposed model empirically achieves the current state-of-the-art results on the IEMOCAP benchmark dataset.
Emotion recognition in multi-party conversation (ERMC) is becoming increasingly popular as an emerging research topic in natural language processing. Prior research focuses on exploring sequential information but ignores the discourse structures of c
onversations. In this paper, we investigate the importance of discourse structures in handling informative contextual cues and speaker-specific features for ERMC. To this end, we propose a discourse-aware graph neural network (ERMC-DisGCN) for ERMC. In particular, we design a relational convolution to lever the self-speaker dependency of interlocutors to propagate contextual information. Furthermore, we exploit a gated convolution to select more informative cues for ERMC from dependent utterances. The experimental results show our method outperforms multiple baselines, illustrating that discourse structures are of great value to ERMC.
Frame semantic parsing is a semantic analysis task based on FrameNet which has received great attention recently. The task usually involves three subtasks sequentially: (1) target identification, (2) frame classification and (3) semantic role labelin
g. The three subtasks are closely related while previous studies model them individually, which ignores their intern connections and meanwhile induces error propagation problem. In this work, we propose an end-to-end neural model to tackle the task jointly. Concretely, we exploit a graph-based method, regarding frame semantic parsing as a graph construction problem. All predicates and roles are treated as graph nodes, and their relations are taken as graph edges. Experiment results on two benchmark datasets of frame semantic parsing show that our method is highly competitive, resulting in better performance than pipeline models.
The capability to automatically detect human stress can benefit artificial intelligent agents involved in affective computing and human-computer interaction. Stress and emotion are both human affective states, and stress has proven to have important
implications on the regulation and expression of emotion. Although a series of methods have been established for multimodal stress detection, limited steps have been taken to explore the underlying inter-dependence between stress and emotion. In this work, we investigate the value of emotion recognition as an auxiliary task to improve stress detection. We propose MUSER -- a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy. Evaluation on the Multimodal Stressed Emotion (MuSE) dataset shows that our model is effective for stress detection with both internal and external auxiliary tasks, and achieves state-of-the-art results.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
