As a fine-grained and local expression behavior measurement, facial action
unit (FAU) analysis (e.g., detection and intensity estimation) has been
documented for its time-consuming, labor-intensive, and error-prone annotation.
Thus a long-standing challenge of FAU analysis arises from the data scarcity of
manual annotations, limiting the generalization ability of trained models to a
large extent. Amounts of previous works have made efforts to alleviate this
issue via semi/weakly supervised methods and extra auxiliary information.
However, these methods still require domain knowledge and have not yet avoided
the high dependency on data annotation. This paper introduces a robust facial
representation model MAE-Face for AU analysis. Using masked autoencoding as the
self-supervised pre-training approach, MAE-Face first learns a high-capacity
model from a feasible collection of face images without additional data
annotations. Then after being fine-tuned on AU datasets, MAE-Face exhibits
convincing performance for both AU detection and AU intensity estimation,
achieving a new state-of-the-art on nearly all the evaluation results. Further
investigation shows that MAE-Face achieves decent performance even when
fine-tuned on only 1\% of the AU training set, strongly proving its robustness
and generalization performance.

本研究介绍了一种使用自监督学习方法的面部表情识别模型 MAE-Face，该模型在只 Fine-tune 1% 的数据集的情况下也可以取得较好的 AU 检测和 AU 强度估计结果.

自监督表示下的面部动作单元检测和强度估计

Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation

Attention mechanism has recently attracted increasing attentions in the field
of facial action unit (AU) detection. By finding the region of interest of each
AU with the attention mechanism, AU-related local features can be captured.
Most of the existing attention based AU detection works use prior knowledge to
predefine fixed attentions or refine the predefined attentions within a small
range, which limits their capacity to model various AUs. In this paper, we
propose an end-to-end deep learning based attention and relation learning
framework for AU detection with only AU labels, which has not been explored
before. In particular, multi-scale features shared by each AU are learned
firstly, and then both channel-wise and spatial attentions are adaptively
learned to select and extract AU-related local features. Moreover, pixel-level
relations for AUs are further captured to refine spatial attentions so as to
extract more relevant local features. Without changing the network
architecture, our framework can be easily extended for AU intensity estimation.
Extensive experiments show that our framework (i) soundly outperforms the
state-of-the-art methods for both AU detection and AU intensity estimation on
the challenging BP4D, DISFA, FERA 2015 and BP4D+ benchmarks, (ii) can
adaptively capture the correlated regions of each AU, and (iii) also works well
under severe occlusions and large poses.

本文提出了一种基于深度学习的端到端注意力关系学习框架，用于面部动作单元（AU）检测。通过自适应地学习通道和空间注意力以选择和提取与 AU 相关的局部特征，并进一步捕获 AU 的像素级关系以提炼更多相关的局部特征，该方法在 AU 检测和强度估计方面均优于目前最先进的方法，并能在严重的遮挡和大姿态下工作。