Multimodal fusion is a significant method for most multimodal tasks. With the
recent surge in the number of large pre-trained models, combining both
multimodal fusion methods and pre-trained model features can achieve
outstanding performance in many multimodal tasks. In this paper, we present our
approach, which leverages both advantages for addressing the task of Expression
(Expr) Recognition and Valence-Arousal (VA) Estimation. We evaluate the
Aff-Wild2 database using pre-trained models, then extract the final hidden
layers of the models as features. Following preprocessing and interpolation or
convolution to align the extracted features, different models are employed for
modal fusion. Our code is available at GitHub - FulgenceWen/ABAW6th.

本文介绍了一种利用多模态融合和预训练模型特征相结合的方法，用于处理表情识别和情绪评估任务，通过对预训练模型提取特征并进行预处理、插值或卷积等操作，并应用不同模型进行多模态融合，达到在多模态任务中出色性能的目的。

情感行为分析中的多模态融合与预训练模型特征研究

Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour  Analysis In-the-wild

The fifth Affective Behavior Analysis in-the-wild (ABAW) Competition is part
of the respective ABAW Workshop which will be held in conjunction with IEEE
Computer Vision and Pattern Recognition Conference (CVPR), 2023. The 5th ABAW
Competition is a continuation of the Competitions held at ECCV 2022, IEEE CVPR
2022, ICCV 2021, IEEE FG 2020 and CVPR 2017 Conferences, and is dedicated at
automatically analyzing affect. For this year's Competition, we feature two
corpora: i) an extended version of the Aff-Wild2 database and ii) the
Hume-Reaction dataset. The former database is an audiovisual one of around 600
videos of around 3M frames and is annotated with respect to:a) two continuous
affect dimensions -valence (how positive/negative a person is) and arousal (how
active/passive a person is)-; b) basic expressions (e.g. happiness, sadness,
neutral state); and c) atomic facial muscle actions (i.e., action units). The
latter dataset is an audiovisual one in which reactions of individuals to
emotional stimuli have been annotated with respect to seven emotional
expression intensities. Thus the 5th ABAW Competition encompasses four
Challenges: i) uni-task Valence-Arousal Estimation, ii) uni-task Expression
Classification, iii) uni-task Action Unit Detection, and iv) Emotional Reaction
Intensity Estimation. In this paper, we present these Challenges, along with
their corpora, we outline the evaluation metrics, we present the baseline
systems and illustrate their obtained performance.

本文介绍了于 CVPR 2023 会议举办的第五届 Affective Behavior Analysis in-the-wild (ABAW) Competition，其中包括四个挑战，使用了两个数据集，分别是 Aff-Wild2 database 和 Hume-Reaction dataset，这些挑战包括单任务的情感估计、表情分类、动作单位检测和情绪反应强度估计。

ABAW：情感价值 - 唤醒评估，表情识别，行为单位检测和情感反应强度评估挑战

ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit  Detection & Emotional Reaction Intensity Estimation Challenges

This paper describes the fourth Affective Behavior Analysis in-the-wild
(ABAW) Competition, held in conjunction with European Conference on Computer
Vision (ECCV), 2022. The 4th ABAW Competition is a continuation of the
Competitions held at IEEE CVPR 2022, ICCV 2021, IEEE FG 2020 and IEEE CVPR 2017
Conferences, and aims at automatically analyzing affect. In the previous runs
of this Competition, the Challenges targeted Valence-Arousal Estimation,
Expression Classification and Action Unit Detection. This year the Competition
encompasses two different Challenges: i) a Multi-Task-Learning one in which the
goal is to learn at the same time (i.e., in a multi-task learning setting) all
the three above mentioned tasks; and ii) a Learning from Synthetic Data one in
which the goal is to learn to recognise the basic expressions from artificially
generated data and generalise to real data. The Aff-Wild2 database is a large
scale in-the-wild database and the first one that contains annotations for
valence and arousal, expressions and action units. This database is the basis
for the above Challenges. In more detail: i) s-Aff-Wild2 -- a static version of
Aff-Wild2 database -- has been constructed and utilized for the purposes of the
Multi-Task-Learning Challenge; and ii) some specific frames-images from the
Aff-Wild2 database have been used in an expression manipulation manner for
creating the synthetic dataset, which is the basis for the Learning from
Synthetic Data Challenge. In this paper, at first we present the two
Challenges, along with the utilized corpora, then we outline the evaluation
metrics and finally present the baseline systems per Challenge, as well as
their derived results. More information regarding the Competition can be found
in the competition's website:
this https URL

本篇论文介绍了在 2022 年欧洲计算机视觉会议 (ECCV) 期间举办的第四届情感行为分析比赛 (ABAW)。今年该比赛涵盖了两个不同的挑战：多任务学习和从合成数据学习，目标在于基于 Aff-Wild2 数据库自动分析情感表达，其中该数据库是一个大规模的野外数据库，具备对价值、唤醒度、表情和动作单位的注释，并通过 s-Aff-Wild2 数据库和人工合成数据集完成了情感识别的基础研究。