Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEmotion-Aware, Emotion-Agnostic, or Automatic: Corpus Creation Strategies to Obtain Cognitive Event Appraisal Annotations
إدراك العاطفة، العاطفة، الأذرع، أو التلقائي: استراتيجيات إنشاء Corpus للحصول على التعليقات التوضيحية تقييم الحدث المعرفي
448
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
توضح نظريات التقييم كيف يؤدي التقييم المعرفي للحدث إلى عاطفة معينة. على النقيض من نظريات المشاعر الأساسية أو التأثير (التكافؤ / الإثارة)، لم تتلق هذه النظرية الكثير من الاهتمام في معالجة اللغة الطبيعية. ومع ذلك، في علم النفس، ثبت أن سميث وإلسنثورث (1985) أظهر أن أبعاد التقييم الاهتمام، اليقين، الجهد المتوقع، والمسؤولية، والمسؤولية السيطرة والتحكم الظرفي تميز بين (على الأقل) 15 فصول العاطفة. ندرس استراتيجيات توضيحية مختلفة لهذه الأبعاد، استنادا إلى كوربوس الفنية التي تركز على الأحداث (Troiano et al.، 2019). نقوم بتحليل اثنين من إعدادات التوضيحية اليدوية: (1) إظهار النص للتعليق أثناء إخفاء ملصق العاطفة ذوي الخبرة؛ (2) الكشف عن العاطفة المرتبطة بالنص. يتيح الإعداد 2 أن يقوم المعلقون بتطوير حدس أكثر واقعية لهذا الحدث الموصوفين، في حين أن الإعداد 1 هو إجراء شروح مزيد من التعريفي، والاعتماد بحت على النص. نقوم بتقييم هذه الاستراتيجيات بطريقتين: من خلال قياس اتفاقية Insine-Annotator وضبط روبرتا للتنبؤ بمتغيرات التقييم. تظهر نتائجنا أن معرفة العاطفة تزيد من موثوقية المعلقين. علاوة على ذلك، نقوم بتقييم استراتيجية وضع العلامات المستندة إلى القواعد التلقائية بحتة (تقييم الاستنتاج من فصول العاطفة المشروحة). يؤدي التدريب على الملصقات المعينة تلقائيا إلى أداء تنافسي من المصنف لدينا، حتى عند اختباره في التوضيحية اليدوية. هذا مؤشر أنه قد يكون من الممكن إنشاء Corpa Corpora تلقائيا لكل مجال موجودا للعاطفة الموجودة بالفعل.
Appraisal theories explain how the cognitive evaluation of an event leads to a particular emotion. In contrast to theories of basic emotions or affect (valence/arousal), this theory has not received a lot of attention in natural language processing. Yet, in psychology it has been proven powerful: Smith and Ellsworth (1985) showed that the appraisal dimensions attention, certainty, anticipated effort, pleasantness, responsibility/control and situational control discriminate between (at least) 15 emotion classes. We study different annotation strategies for these dimensions, based on the event-focused enISEAR corpus (Troiano et al., 2019). We analyze two manual annotation settings: (1) showing the text to annotate while masking the experienced emotion label; (2) revealing the emotion associated with the text. Setting 2 enables the annotators to develop a more realistic intuition of the described event, while Setting 1 is a more standard annotation procedure, purely relying on text. We evaluate these strategies in two ways: by measuring inter-annotator agreement and by fine- tuning RoBERTa to predict appraisal variables. Our results show that knowledge of the emotion increases annotators' reliability. Further, we evaluate a purely automatic rule-based labeling strategy (inferring appraisal from annotated emotion classes). Training on automatically assigned labels leads to a competitive performance of our classifier, even when tested on manual annotations. This is an indicator that it might be possible to automatically create appraisal corpora for every domain for which emotion corpora already exist.
References used
https://aclanthology.org/
rate research
Read More
Emotion recognition in multi-party conversation (ERMC) is becoming increasingly popular as an emerging research topic in natural language processing. Prior research focuses on exploring sequential information but ignores the discourse structures of c
onversations. In this paper, we investigate the importance of discourse structures in handling informative contextual cues and speaker-specific features for ERMC. To this end, we propose a discourse-aware graph neural network (ERMC-DisGCN) for ERMC. In particular, we design a relational convolution to lever the self-speaker dependency of interlocutors to propagate contextual information. Furthermore, we exploit a gated convolution to select more informative cues for ERMC from dependent utterances. The experimental results show our method outperforms multiple baselines, illustrating that discourse structures are of great value to ERMC.
Song lyrics convey a multitude of emotions to the listener and powerfully portray the emotional state of the writer or singer. This paper examines a variety of modeling approaches to the multi-emotion classification problem for songs. We introduce th
e Edmonds Dance dataset, a novel emotion-annotated lyrics dataset from the reader's perspective, and annotate the dataset of Mihalcea and Strapparava (2012) at the song level. We find that models trained on relatively small song datasets achieve marginally better performance than BERT (Devlin et al., 2018) fine-tuned on large social media or dialog datasets.
We present a model to predict fine-grained emotions along the continuous dimensions of valence, arousal, and dominance (VAD) with a corpus with categorical emotion annotations. Our model is trained by minimizing the EMD (Earth Mover's Distance) loss
between the predicted VAD score distribution and the categorical emotion distributions sorted along VAD, and it can simultaneously classify the emotion categories and predict the VAD scores for a given sentence. We use pre-trained RoBERTa-Large and fine-tune on three different corpora with categorical labels and evaluate on EmoBank corpus with VAD scores. We show that our approach reaches comparable performance to that of the state-of-the-art classifiers in categorical emotion classification and shows significant positive correlations with the ground truth VAD scores. Also, further training with supervision of VAD labels leads to improved performance especially when dataset is small. We also present examples of predictions of appropriate emotion words that are not part of the original annotations.
When humans judge the affective content of texts, they also implicitly assess the correctness of such judgment, that is, their confidence. We hypothesize that people's (in)confidence that they performed well in an annotation task leads to (dis)agreem
ents among each other. If this is true, confidence may serve as a diagnostic tool for systematic differences in annotations. To probe our assumption, we conduct a study on a subset of the Corpus of Contemporary American English, in which we ask raters to distinguish neutral sentences from emotion-bearing ones, while scoring the confidence of their answers. Confidence turns out to approximate inter-annotator disagreements. Further, we find that confidence is correlated to emotion intensity: perceiving stronger affect in text prompts annotators to more certain classification performances. This insight is relevant for modelling studies of intensity, as it opens the question wether automatic regressors or classifiers actually predict intensity, or rather human's self-perceived confidence.
In deployment, systems that use speech as input must make use of automated transcriptions. Yet, typically when these systems are evaluated, gold transcriptions are assumed. We explicitly examine the impact of transcription errors on the downstream pe
rformance of a multi-modal system on three related tasks from three datasets: emotion, sarcasm, and personality detection. We include three separate transcription tools and show that while all automated transcriptions propagate errors that substantially impact downstream performance, the open-source tools fair worse than the paid tool, though not always straightforwardly, and word error rates do not correlate well with downstream performance. We further find that the inclusion of audio features partially mitigates transcription errors, but that a naive usage of a multi-task setup does not.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
