Facial expression is related to facial muscle contractions and different
muscle movements correspond to different emotional states. For micro-expression
recognition, the muscle movements are usually subtle, which has a negative
impact on the performance of current facial emotion recognition algorithms.
Most existing methods use self-attention mechanisms to capture relationships
between tokens in a sequence, but they do not take into account the inherent
spatial relationships between facial landmarks. This can result in sub-optimal
performance on micro-expression recognition tasks.Therefore, learning to
recognize facial muscle movements is a key challenge in the area of
micro-expression recognition. In this paper, we propose a Hierarchical
Transformer Network (HTNet) to identify critical areas of facial muscle
movement. HTNet includes two major components: a transformer layer that
leverages the local temporal features and an aggregation layer that extracts
local and global semantical facial features. Specifically, HTNet divides the
face into four different facial areas: left lip area, left eye area, right eye
area and right lip area. The transformer layer is used to focus on representing
local minor muscle movement with local self-attention in each area. The
aggregation layer is used to learn the interactions between eye areas and lip
areas. The experiments on four publicly available micro-expression datasets
show that the proposed approach outperforms previous methods by a large margin.
The codes and models are available at:
https://github.com/wangzhifengharrison/HTNet

面部表情与肌肉收缩相关，微表情识别中，肌肉运动通常是微妙的，对当前面部情感识别算法性能产生负面影响。本文提出了一种层次 Transformer 网络 (HTNet) 来识别关键的面部肌肉运动区域，通过局部时间特征和全局面部特征提取层组成。实验证明，该方法在四个公开的微表情数据集上优于以前的方法并提供了相关代码和模型。