Human affective behavior analysis aims to delve into human expressions and
behaviors to deepen our understanding of human emotions. Basic expression
categories (EXPR) and Action Units (AUs) are two essential components in this
analysis, which categorize emotions and break down facial movements into
elemental units, respectively. Despite advancements, existing approaches in
expression classification and AU detection often necessitate complex models and
substantial computational resources, limiting their applicability in everyday
settings. In this work, we introduce the first lightweight framework adept at
efficiently tackling both expression classification and AU detection. This
framework employs a frozen CLIP image encoder alongside a trainable multilayer
perceptron (MLP), enhanced with Conditional Value at Risk (CVaR) for robustness
and a loss landscape flattening strategy for improved generalization.
Experimental results on the Aff-wild2 dataset demonstrate superior performance
in comparison to the baseline while maintaining minimal computational demands,
offering a practical solution for affective behavior analysis. The code is
available at this https URL

人类情感行为分析的研究旨在深入了解人类情绪，其中表情分类、动作单位是其两个主要组成部分。本研究提出了一种轻量级框架，可以高效地处理表情分类和动作单位检测，通过采用冻结的 CLIP 图像编码器和可训练的多层感知机（MLP），以 CVaR 提升稳健性，并采用损失函数平滑策略来改进泛化性能，在 Aff-wild2 数据集上得到了优越的性能表现，同时保持了最小的计算需求，提供了一种实用的情感行为分析解决方案。

具有 CLIP 的鲁棒轻量级面部情感行为识别

Robust Light-Weight Facial Affective Behavior Recognition with CLIP

For computers to recognize human emotions, expression classification is an
equally important problem in the human-computer interaction area. In the 3rd
Affective Behavior Analysis In-The-Wild competition, the task of expression
classification includes eight classes with six basic expressions of human faces
from videos. In this paper, we employ a transformer mechanism to encode the
robust representation from the backbone. Fusion of the robust representations
plays an important role in the expression classification task. Our approach
achieves 30.35\% and 28.60\% for the $F_1$ score on the validation set and the
test set, respectively. This result shows the effectiveness of the proposed
architecture based on the Aff-Wild2 dataset.

本研究使用 Transformer 机制，结合鲁棒表示编码和表示融合进行表情分类，结果表明该模型在 Aff-Wild2 数据集上表现有效。