3D facial landmark localization has proven to be of particular use for
applications, such as face tracking, 3D face modeling, and image-based 3D face
reconstruction. In the supervised learning case, such methods usually rely on
3D landmark datasets derived from 3DMM-based registration that often lack
spatial definition alignment, as compared with that chosen by hand-labeled
human consensus, e.g., how are eyebrow landmarks defined? This creates a gap
between landmark datasets generated via high-quality 2D human labels and 3DMMs,
and it ultimately limits their effectiveness. To address this issue, we
introduce a novel semi-supervised learning approach that learns 3D landmarks by
directly lifting (visible) hand-labeled 2D landmarks and ensures better
definition alignment, without the need for 3D landmark datasets. To lift 2D
landmarks to 3D, we leverage 3D-aware GANs for better multi-view consistency
learning and in-the-wild multi-frame videos for robust cross-generalization.
Empirical experiments demonstrate that our method not only achieves better
definition alignment between 2D-3D landmarks but also outperforms other
supervised learning 3D landmark localization methods on both 3DMM labeled and
photogrammetric ground truth evaluation datasets. Project Page:
this https URL

介绍了一种采用半监督学习方法，在没有 3D 地标数据集的情况下，通过直接提取（可见的）手工标记的 2D 地标，并确保更好的定义对齐，从而学习 3D 地标。该方法利用 3D 感知的生成对抗网络进行更好的多视角一致性学习，并利用实景多帧视频进行鲁棒的跨领域泛化。实证实验证明，该方法不仅在 2D-3D 地标之间实现了更好的定义对齐，还在 3DMM 标记和摄影测量基准数据集上优于其他监督学习的 3D 地标定位方法。