Human affective behavior analysis aims to delve into human expressions and
behaviors to deepen our understanding of human emotions. Basic expression
categories (EXPR) and Action Units (AUs) are two essential components in this
analysis, which categorize emotions and break down facial movements into
elemental units, respectively. Despite advancements, existing approaches in
expression classification and AU detection often necessitate complex models and
substantial computational resources, limiting their applicability in everyday
settings. In this work, we introduce the first lightweight framework adept at
efficiently tackling both expression classification and AU detection. This
framework employs a frozen CLIP image encoder alongside a trainable multilayer
perceptron (MLP), enhanced with Conditional Value at Risk (CVaR) for robustness
and a loss landscape flattening strategy for improved generalization.
Experimental results on the Aff-wild2 dataset demonstrate superior performance
in comparison to the baseline while maintaining minimal computational demands,
offering a practical solution for affective behavior analysis. The code is
available at this https URL

人类情感行为分析的研究旨在深入了解人类情绪，其中表情分类、动作单位是其两个主要组成部分。本研究提出了一种轻量级框架，可以高效地处理表情分类和动作单位检测，通过采用冻结的 CLIP 图像编码器和可训练的多层感知机（MLP），以 CVaR 提升稳健性，并采用损失函数平滑策略来改进泛化性能，在 Aff-wild2 数据集上得到了优越的性能表现，同时保持了最小的计算需求，提供了一种实用的情感行为分析解决方案。

具有 CLIP 的鲁棒轻量级面部情感行为识别

Robust Light-Weight Facial Affective Behavior Recognition with CLIP

Deep learning-based models are at the forefront of most driver observation
benchmarks due to their remarkable accuracies but are also associated with high
computational costs. This is challenging, as resources are often limited in
real-world driving scenarios. This paper introduces a lightweight framework for
resource-efficient driver activity recognition. The framework enhances 3D
MobileNet, a neural architecture optimized for speed in video classification,
by incorporating knowledge distillation and model quantization to balance model
accuracy and computational efficiency. Knowledge distillation helps maintain
accuracy while reducing the model size by leveraging soft labels from a larger
teacher model (I3D), instead of relying solely on original ground truth data.
Model quantization significantly lowers memory and computation demands by using
lower precision integers for model weights and activations. Extensive testing
on a public dataset for in-vehicle monitoring during autonomous driving
demonstrates that this new framework achieves a threefold reduction in model
size and a 1.4-fold improvement in inference time, compared to an already
optimized architecture. The code for this study is available at
this https URL

通过使用知识蒸馏和模型量化来增强 3D MobileNet，该论文介绍了一个轻量级的资源高效驾驶员活动识别框架，通过从较大的教师模型（I3D）获取软标签来降低模型大小，降低内存和计算需求，从而实现模型大小的减小三倍和推断时间提升 1.4 倍。