The emotion detection technology to enhance human decision-making is an
important research issue for real-world applications, but real-life emotion
datasets are relatively rare and small. The experiments conducted in this paper
use the CEMO, which was collected in a French emergency call center. Two
pre-trained models based on speech and text were fine-tuned for speech emotion
recognition. Using pre-trained Transformer encoders mitigates our data's
limited and sparse nature. This paper explores the different fusion strategies
of these modality-specific models. In particular, fusions with and without
cross-attention mechanisms were tested to gather the most relevant information
from both the speech and text encoders. We show that multimodal fusion brings
an absolute gain of 4-9% with respect to either single modality and that the
Symmetric multi-headed cross-attention mechanism performed better than late
classical fusion approaches. Our experiments also suggest that for the
real-life CEMO corpus, the audio component encodes more emotive information
than the textual one.

本文探讨了基于 CEMO 数据集的语音和文本预训练模型在情感识别上的融合策略，研究表明基于多模态融合的交叉关注机制实现了更好的表现。同时，实验也表明在 CEMO 上，音频编码包含的情感信息比文本更为丰富。

探索应用于紧急呼叫中心语料库的多模态情感识别的注意力机制

Exploring Attention Mechanisms for Multimodal Emotion Recognition in an  Emergency Call Center Corpus

The development of emotion detection technology has emerged as a highly
valuable possibility in the corporate sector due to the nearly limitless uses
of this new discipline, particularly with the unceasing propagation of social
data. In recent years, the electronic marketplace has witnessed the
establishment of a large number of start-up businesses with an almost sole
focus on building new commercial and open-source tools and APIs for emotion
detection and recognition. Yet, these tools and APIs must be continuously
reviewed and evaluated, and their performances should be reported and
discussed. There is a lack of research to empirically compare current emotion
detection technologies in terms of the results obtained from each model using
the same textual dataset. Also, there is a lack of comparative studies that
apply benchmark comparison to social data. This study compares eight
technologies; IBM Watson NLU, ParallelDots, Symanto-Ekman, Crystalfeel, Text to
Emotion, Senpy, Textprobe, and NLP Cloud. The comparison was undertaken using
two different datasets. The emotions from the chosen datasets were then derived
using the incorporated APIs. The performance of these APIs was assessed using
the aggregated scores that they delivered as well as the theoretically proven
evaluation metrics such as the micro-average of accuracy, classification error,
precision, recall, and f1-score. Lastly, the assessment of these APIs
incorporating the evaluation measures is reported and discussed.

本研究对 IBM Watson NLU、ParallelDots、Symanto-Ekman、Crystalfeel、Text to Emotion、Senpy、Textprobe 和 NLP Cloud 等八种情感检测技术进行了比较，使用两个不同的数据集进行了比较，并使用理论证明的评估指标对这些 API 的性能进行了评估和讨论。