In this paper, we study different approaches for classifying emotions from
speech using acoustic and text-based features. We propose to obtain
contextualized word embeddings with BERT to represent the information contained
in speech transcriptions and show that this results in better performance than
using Glove embeddings. We also propose and compare different strategies to
combine the audio and text modalities, evaluating them on IEMOCAP and
MSP-PODCAST datasets. We find that fusing acoustic and text-based systems is
beneficial on both datasets, though only subtle differences are observed across
the evaluated fusion approaches. Finally, for IEMOCAP, we show the large effect
that the criteria used to define the cross-validation folds have on results. In
particular, the standard way of creating folds for this dataset results in a
highly optimistic estimation of performance for the text-based system,
suggesting that some previous works may overestimate the advantage of
incorporating transcriptions.

使用声学和文本特征，研究通过不同方法对语言情感进行分类的研究。使用 BERT 获取情感转录中所包含信息的上下文化词嵌入，相比使用 Glove 嵌入，结果表明表现更好。我们还提出并比较不同的音频和文本模式结合策略，并在 IEMOCAP 和 MSP-PODCAST 数据集上进行评估。我们发现，在这两个数据集上，融合声学和文本系统是有益的，尽管在评估的融合方法中只观察到微小的差异。最后，对于 IEMOCAP 数据集，我们展示了用于定义交叉验证折叠的标准方法对结果的巨大影响。特别是，为该数据集创建折叠的标准方法导致对于文本系统的性能高度乐观的估计，这表明一些先前的作品可能夸大了结合转录的优势。

融合声学和基于文本特征的情感识别的方法

Fusion approaches for emotion recognition from speech using acoustic and  text-based features

Neural networks for computer vision extract uninterpretable features despite
achieving high accuracy on benchmarks. In contrast, humans can explain their
predictions using succinct and intuitive descriptions. To incorporate
explainability into neural networks, we train a vision model whose feature
representations are text. We show that such a model can effectively classify
ImageNet images, and we discuss the challenges we encountered when training it.

本研究训练了一个视觉模型，使用基于文本的特征表征图像，以达到在图像分类方面有效且具有可解释性的成果，进一步探讨了训练过程中遇到的挑战。