Speech Emotion Recognition (SER) affective technology enables the intelligent
embedded devices to interact with sensitivity. Similarly, call centre employees
recognise customers' emotions from their pitch, energy, and tone of voice so as
to modify their speech for a high-quality interaction with customers. This work
explores, for the first time, the effects of the harmonic and percussive
components of Mel spectrograms in SER. We attempt to leverage the Mel
spectrogram by decomposing distinguishable acoustic features for exploitation
in our proposed architecture, which includes a novel feature map generator
algorithm, a CNN-based network feature extractor and a multi-layer perceptron
(MLP) classifier. This study specifically focuses on effective data
augmentation techniques for building an enriched hybrid-based feature map. This
process results in a function that outputs a 2D image so that it can be used as
input data for a pre-trained CNN-VGG16 feature extractor. Furthermore, we also
investigate other acoustic features such as MFCCs, chromagram, spectral
contrast, and the tonnetz to assess our proposed framework. A test accuracy of
92.79% on the Berlin EMO-DB database is achieved. Our result is higher than
previous works using CNN-VGG16.

该研究通过对 Mel 谱图的谐波和打击性组成部分进行分析，提出了一个新的架构，包括特征映射生成器算法、基于 CNN 的网络特征提取器和多层感知器（MLP）分类器，研究有效的数据增强技术建立了丰富的混合特征映射，最终在 Berlin EMO-DB 数据库上取得了 92.79% 的测试准确率，结果优于之前使用 CNN-VGG16 的研究。