Since Facial Action Unit (AU) annotations require domain expertise, common AU
datasets only contain a limited number of subjects. As a result, a crucial
challenge for AU detection is addressing identity overfitting. We find that AUs
and facial expressions are highly associated, and existing facial expression
datasets often contain a large number of identities. In this paper, we aim to
utilize the expression datasets without AU labels to facilitate AU detection.
Specifically, we develop a novel AU detection framework aided by the
Global-Local facial Expressions Embedding, dubbed GLEE-Net. Our GLEE-Net
consists of three branches to extract identity-independent expression features
for AU detection. We introduce a global branch for modeling the overall facial
expression while eliminating the impacts of identities. We also design a local
branch focusing on specific local face regions. The combined output of global
and local branches is firstly pre-trained on an expression dataset as an
identity-independent expression embedding, and then finetuned on AU datasets.
Therefore, we significantly alleviate the issue of limited identities.
Furthermore, we introduce a 3D global branch that extracts expression
coefficients through 3D face reconstruction to consolidate 2D AU descriptions.
Finally, a Transformer-based multi-label classifier is employed to fuse all the
representations for AU detection. Extensive experiments demonstrate that our
method significantly outperforms the state-of-the-art on the widely-used DISFA,
BP4D and BP4D+ datasets.

本研究旨在使用没有 AU 标签的表情数据集来促进 AU 检测，开发了一种名为 GLEE-Net 的新型 AU 检测框架，其中包括三个分支来提取独立于身份的表情特征，并通过 Transformer 多标签分类器来融合所有表示以进行 AU 检测。实验表明，该方法在 DISFA，BP4D 和 BP4D + 数据集上显著优于目前的最先进技术。