3D facial landmark localization has proven to be of particular use for
applications, such as face tracking, 3D face modeling, and image-based 3D face
reconstruction. In the supervised learning case, such methods usually rely on
3D landmark datasets derived from 3DMM-based registration that often lack
spatial definition alignment, as compared with that chosen by hand-labeled
human consensus, e.g., how are eyebrow landmarks defined? This creates a gap
between landmark datasets generated via high-quality 2D human labels and 3DMMs,
and it ultimately limits their effectiveness. To address this issue, we
introduce a novel semi-supervised learning approach that learns 3D landmarks by
directly lifting (visible) hand-labeled 2D landmarks and ensures better
definition alignment, without the need for 3D landmark datasets. To lift 2D
landmarks to 3D, we leverage 3D-aware GANs for better multi-view consistency
learning and in-the-wild multi-frame videos for robust cross-generalization.
Empirical experiments demonstrate that our method not only achieves better
definition alignment between 2D-3D landmarks but also outperforms other
supervised learning 3D landmark localization methods on both 3DMM labeled and
photogrammetric ground truth evaluation datasets. Project Page:
this https URL

介绍了一种采用半监督学习方法，在没有 3D 地标数据集的情况下，通过直接提取（可见的）手工标记的 2D 地标，并确保更好的定义对齐，从而学习 3D 地标。该方法利用 3D 感知的生成对抗网络进行更好的多视角一致性学习，并利用实景多帧视频进行鲁棒的跨领域泛化。实证实验证明，该方法不仅在 2D-3D 地标之间实现了更好的定义对齐，还在 3DMM 标记和摄影测量基准数据集上优于其他监督学习的 3D 地标定位方法。

FaceLift: 半监督的三维面部关键点定位

FaceLift: Semi-supervised 3D Facial Landmark Localization

3D face shape is more expressive and viewpoint-consistent than its 2D
counterpart. However, 3D facial landmark localization in a single image is
challenging due to the ambiguous nature of landmarks under 3D perspective.
Existing approaches typically adopt a suboptimal two-step strategy, performing
2D landmark localization followed by depth estimation. In this paper, we
propose the Joint Voxel and Coordinate Regression (JVCR) method for 3D facial
landmark localization, addressing it more effectively in an end-to-end fashion.
First, a compact volumetric representation is proposed to encode the per-voxel
likelihood of positions being the 3D landmarks. The dimensionality of such a
representation is fixed regardless of the number of target landmarks, so that
the curse of dimensionality could be avoided. Then, a stacked hourglass network
is adopted to estimate the volumetric representation from coarse to fine,
followed by a 3D convolution network that takes the estimated volume as input
and regresses 3D coordinates of the face shape. In this way, the 3D structural
constraints between landmarks could be learned by the neural network in a more
efficient manner. Moreover, the proposed pipeline enables end-to-end training
and improves the robustness and accuracy of 3D facial landmark localization.
The effectiveness of our approach is validated on the 3DFAW and AFLW2000-3D
datasets. Experimental results show that the proposed method achieves
state-of-the-art performance in comparison with existing methods.

本文提出了一种基于 Joint Voxel and Coordinate Regression（JVCR）的 3D 面部标志定位方法，该方法采用端到端的方式，通过稠密的容积表示和卷积神经网络实现了更高效的学习和更准确的结果。该方法在 3DFAW 和 AFLW2000-3D 数据集上达到了最先进的性能。