Automatic emotion recognition plays a significant role in the process of
human computer interaction and the design of Internet of Things (IOT)
technologies. Yet, a common problem in emotion recognition systems lies in the
scarcity of reliable labels. By modeling pairwise differences between samples
of interest, a Siamese network can help to mitigate this challenge since it
requires fewer samples than traditional deep learning methods. In this paper,
we propose a distance loss, which can be applied on the Siamese network
fine-tuning, by optimizing the model based on the relevant distance between
same and difference class pairs. Our system use samples from the source data to
pre-train the weights of proposed Siamese neural network, which are fine-tuned
based on the target data. We present an emotion recognition task that uses
speech, since it is one of the most ubiquitous and frequently used
bio-behavioral signals. Our target data comes from the RAVDESS dataset, while
the CREMA-D and eNTERFACE'05 are used as source data, respectively. Our results
indicate that the proposed distance loss is able to greatly benefit the
fine-tuning process of Siamese network. Also, the selection of source data has
more effect on the Siamese network performance compared to the number of frozen
layers. These suggest the great potential of applying the Siamese network and
modelling pairwise differences in the field of transfer learning for automatic
emotion recognition.

使用 Siamese 网络进行跨语音情感识别的迁移学习，通过 fine-tuning 和优化 distance loss 可以提高识别准确性。