Recently, heatmap regression models have become popular due to their superior
performance in locating facial landmarks. However, three major problems still
exist among these models: (1) they are computationally expensive; (2) they
usually lack explicit constraints on global shapes; (3) domain gaps are
commonly present. To address these problems, we propose Pixel-in-Pixel Net
(PIPNet) for facial landmark detection. The proposed model is equipped with a
novel detection head based on heatmap regression, which conducts score and
offset predictions simultaneously on low-resolution feature maps. By doing so,
repeated upsampling layers are no longer necessary, enabling the inference time
to be largely reduced without sacrificing model accuracy. Besides, a simple but
effective neighbor regression module is proposed to enforce local constraints
by fusing predictions from neighboring landmarks, which enhances the robustness
of the new detection head. To further improve the cross-domain generalization
capability of PIPNet, we propose self-training with curriculum. This training
strategy is able to mine more reliable pseudo-labels from unlabeled data across
domains by starting with an easier task, then gradually increasing the
difficulty to provide more precise labels. Extensive experiments demonstrate
the superiority of PIPNet, which obtains state-of-the-art results on three out
of six popular benchmarks under the supervised setting. The results on two
cross-domain test sets are also consistently improved compared to the
baselines. Notably, our lightweight version of PIPNet runs at 35.7 FPS and 200
FPS on CPU and GPU, respectively, while still maintaining a competitive
accuracy to state-of-the-art methods. The code of PIPNet is available at
this https URL

本文提出了 Pixel-in-Pixel Net（PIPNet） 对面部关键点检测的改进方法，旨在解决热图回归模型在计算效率、全局模型约束和域差异等方面的问题。实验表明，该模型在三个受监督学习的基准测试中均具有最先进的表现，而在两个跨领域测试集上的结果也得到了明显改善。同时，PIPNet 的轻量级版本在 CPU 和 GPU 上分别以 35.7 和 200 FPS 的速度运行，仍然保持了与最先进方法的竞争精度。