Self-supervised pre-training has been proved to be effective in learning
transferable representations that benefit various visual tasks. This paper asks
this question: can self-supervised pre-training learn general facial
representations for various facial analysis tasks? Recent efforts toward this
goal are limited to treating each face image as a whole, i.e., learning
consistent facial representations at the image-level, which overlooks the
consistency of local facial representations (i.e., facial regions like eyes,
nose, etc). In this work, we make a first attempt to propose a novel
self-supervised facial representation learning framework to learn consistent
global and local facial representations, Facial Region Awareness (FRA).
Specifically, we explicitly enforce the consistency of facial regions by
matching the local facial representations across views, which are extracted
with learned heatmaps highlighting the facial regions. Inspired by the mask
prediction in supervised semantic segmentation, we obtain the heatmaps via
cosine similarity between the per-pixel projection of feature maps and facial
mask embeddings computed from learnable positional embeddings, which leverage
the attention mechanism to globally look up the facial image for facial
regions. To learn such heatmaps, we formulate the learning of facial mask
embeddings as a deep clustering problem by assigning the pixel features from
the feature maps to them. The transfer learning results on facial
classification and regression tasks show that our FRA outperforms previous
pre-trained models and more importantly, using ResNet as the unified backbone
for various tasks, our FRA achieves comparable or even better performance
compared with SOTA methods in facial analysis tasks.

本研究提出了一种新型的自监督人脸表征学习框架 (Facial Region Awareness, FRA)，目标是通过匹配面部局部表征来学习一致的全局和局部面部表征，并通过在特征图上计算面部蒙版嵌入的余弦相似度来获取热图，进而证明在面部分类和回归任务中，FRA 模型优于之前的预训练模型和现有方法。