Human emotion recognition holds a pivotal role in facilitating seamless
human-computer interaction. This paper delineates our methodology in tackling
the Valence-Arousal (VA) Estimation Challenge, Expression (Expr) Classification
Challenge, and Action Unit (AU) Detection Challenge within the ambit of the 6th
Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Our
study advocates a novel approach aimed at refining continuous emotion
recognition. We achieve this by initially harnessing pre-training with Masked
Autoencoders (MAE) on facial datasets, followed by fine-tuning on the aff-wild2
dataset annotated with expression (Expr) labels. The pre-trained model serves
as an adept visual feature extractor, thereby enhancing the model's robustness.
Furthermore, we bolster the performance of continuous emotion recognition by
integrating Temporal Convolutional Network (TCN) modules and Transformer
Encoder modules into our framework.

本研究以人类情绪识别为核心，通过采用面部数据集上的 Masked Autoencoders (MAE) 预训练和 aff-wild2 数据集的微调，结合 Temporal Convolutional Network (TCN) 模块和 Transformer Encoder 模块，实现了对情绪的连续识别。

利用掩模自编码器、时域卷积网络和转换器进行连续情感识别的自预训练强化

Boosting Continuous Emotion Recognition with Self-Pretraining using  Masked Autoencoders, Temporal Convolutional Networks, and Transformers

Pre-training for feature extraction is an increasingly studied approach to
get better continuous representations of audio and text content. In the present
work, we use wav2vec and camemBERT as self-supervised learned models to
represent our data in order to perform continuous emotion recognition from
speech (SER) on AlloSat, a large French emotional database describing the
satisfaction dimension, and on the state of the art corpus SEWA focusing on
valence, arousal and liking dimensions. To the authors' knowledge, this paper
presents the first study showing that the joint use of wav2vec and BERT-like
pre-trained features is very relevant to deal with continuous SER task, usually
characterized by a small amount of labeled training data. Evaluated by the
well-known concordance correlation coefficient (CCC), our experiments show that
we can reach a CCC value of 0.825 instead of 0.592 when using MFCC in
conjunction with word2vec word embedding on the AlloSat dataset.

本文研究音频和文本的预训练提取特征方法，使用 wav2vec 和 camemBERT 模型进行连续情感识别任务，同时采用 SEWA 数据集，证明 wav2vec 和 BERT 预训练特征的联合使用在处理连续 SER 任务中表现非常合适。通过实验证明，新方法的 CCC 值达到 0.825，而传统的 MFCC 和 word2vec 仅为 0.592。