Supervised keypoint localization methods rely on large manually labeled image
datasets, where objects can deform, articulate, or occlude. However, creating
such large keypoint labels is time-consuming and costly, and is often
error-prone due to inconsistent labeling. Thus, we desire an approach that can
learn keypoint localization with fewer yet consistently annotated images. To
this end, we present a novel formulation that learns to localize semantically
consistent keypoint definitions, even for occluded regions, for varying object
categories. We use a few user-labeled 2D images as input examples, which are
extended via self-supervision using a larger unlabeled dataset. Unlike
unsupervised methods, the few-shot images act as semantic shape constraints for
object localization. Furthermore, we introduce 3D geometry-aware constraints to
uplift keypoints, achieving more accurate 2D localization. Our general-purpose
formulation paves the way for semantically conditioned generative modeling and
attains competitive or state-of-the-art accuracy on several datasets, including
human faces, eyes, animals, cars, and never-before-seen mouth interior (teeth)
localization tasks, not attempted by the previous few-shot methods. Project
page:
https://xingzhehe.github.io/FewShot3DKP/}{this https URL

本文提出了一种基于 Few-shot learning 的填充关键点标注鲁棒性方法，通过少数的半监督数据和大量的自监督数据，在各种不同的物体类别中实现了语义一致的关键点定位和更准确的 3D 几何重建局部特征的二维定位。