The activations of Facial Action Units (AUs) mutually influence one another.
While the relationship between a pair of AUs can be complex and unique,
existing approaches fail to specifically and explicitly represent such cues for
each pair of AUs in each facial display. This paper proposes an AU relationship
modelling approach that deep learns a unique graph to explicitly describe the
relationship between each pair of AUs of the target facial display. Our
approach first encodes each AU's activation status and its association with
other AUs into a node feature. Then, it learns a pair of multi-dimensional edge
features to describe multiple task-specific relationship cues between each pair
of AUs. During both node and edge feature learning, our approach also considers
the influence of the unique facial display on AUs' relationship by taking the
full face representation as an input. Experimental results on BP4D and DISFA
datasets show that both node and edge feature learning modules provide large
performance improvements for CNN and transformer-based backbones, with our best
systems achieving the state-of-the-art AU recognition results. Our approach not
only has a strong capability in modelling relationship cues for AU recognition
but also can be easily incorporated into various backbones. Our PyTorch code is
made available.

本文提出了一种基于深度学习的 AU 关系建模方法，通过学习多维边特征与节点特征来描述面部表情不同组合之间的关系特征，从而实现更准确的 AU 识别。我们在 BP4D 和 DISFA 数据集上的实验结果表明，我们的方法不仅在各项识别指标上取得最优结果，而且可以轻松嵌入到不同的模型中。