A multi-modal emotional speech Mandarin database including articulatory
kinematics, acoustics, glottal and facial micro-expressions is designed and
established, which is described in detail from the aspects of corpus design,
subject selection, recording details and data processing. Where signals are
labeled with discrete emotion labels (neutral, happy, pleasant, indifferent,
angry, sad, grief) and dimensional emotion labels (pleasure, arousal,
dominance). In this paper, the validity of dimension annotation is verified by
statistical analysis of dimension annotation data. The SCL-90 scale data of
annotators are verified and combined with PAD annotation data for analysis, so
as to explore the internal relationship between the outlier phenomenon in
annotation and the psychological state of annotators. In order to verify the
speech quality and emotion discrimination of the database, this paper uses 3
basic models of SVM, CNN and DNN to calculate the recognition rate of these
seven emotions. The results show that the average recognition rate of seven
emotions is about 82% when using acoustic data alone. When using glottal data
alone, the average recognition rate is about 72%. Using kinematics data alone,
the average recognition rate also reaches 55.7%. Therefore, the database is of
high quality and can be used as an important source for speech analysis
research, especially for the task of multimodal emotional speech analysis.

设计并建立了一个包括发音运动学、声学、声门和面部微表情的多模态情感汉语数据库，描述了从语料库设计、被试选择、录音细节和数据处理等方面的详细信息。通过对维度注释数据的统计分析验证了维度注释的有效性。使用支持向量机、卷积神经网络和深度神经网络计算了这七种情感的识别率，结果显示仅使用声学数据识别七种情感的平均识别率约为 82%。因此，该数据库具有高质量，可用作重要的语音分析研究来源，特别是用于多模态情感语音分析任务。