We present Supervision by Registration and Triangulation (SRT), an
unsupervised approach that utilizes unlabeled multi-view video to improve the
accuracy and precision of landmark detectors. Being able to utilize unlabeled
data enables our detectors to learn from massive amounts of unlabeled data
freely available and not be limited by the quality and quantity of manual human
annotations. To utilize unlabeled data, there are two key observations: (1) the
detections of the same landmark in adjacent frames should be coherent with
registration, i.e., optical flow. (2) the detections of the same landmark in
multiple synchronized and geometrically calibrated views should correspond to a
single 3D point, i.e., multi-view consistency. Registration and multi-view
consistency are sources of supervision that do not require manual labeling,
thus it can be leveraged to augment existing training data during detector
training. End-to-end training is made possible by differentiable registration
and 3D triangulation modules. Experiments with 11 datasets and a newly proposed
metric to measure precision demonstrate accuracy and precision improvements in
landmark detection on both images and video. Code is available at
this https URL

本文介绍了一种无人监督的方法，使用未标记的多视角视频来改善地标探测器的精度和准确性，利用不需要人工注释的监督源，通过可微的配准和三维三角测量模块进行端对端训练，实验表明该方法可以提高地标检测在图像和视频中的精度和准确性。

通过注册和三角化进行地标检测的监督

Supervision by Registration and Triangulation for Landmark Detection

Equivariance to random image transformations is an effective method to learn
landmarks of object categories, such as the eyes and the nose in faces, without
manual supervision. However, this method does not explicitly guarantee that the
learned landmarks are consistent with changes between different instances of
the same object, such as different facial identities. In this paper, we develop
a new perspective on the equivariance approach by noting that dense landmark
detectors can be interpreted as local image descriptors equipped with
invariance to intra-category variations. We then propose a direct method to
enforce such an invariance in the standard equivariant loss. We do so by
exchanging descriptor vectors between images of different object instances
prior to matching them geometrically. In this manner, the same vectors must
work regardless of the specific object identity considered. We use this
approach to learn vectors that can simultaneously be interpreted as local
descriptors and dense landmarks, combining the advantages of both. Experiments
on standard benchmarks show that this approach can match, and in some cases
surpass state-of-the-art performance amongst existing methods that learn
landmarks without supervision. Code is available at
www.robots.ox.ac.uk/~vgg/research/DVE/.

本论文研究对象类别的标志物，提出使用等变性法则与局部图像描述符相结合的方法，使得不同对象实例之间学习到的标记物是具有一致性的。实验表明该方法能够超越现有无监督学习方法的表现。

通过描述向量交换进行无监督地标学习

Unsupervised Learning of Landmarks by Descriptor Vector Exchange

We propose a method for learning landmark detectors for visual objects (such
as the eyes and the nose in a face) without any manual supervision. We cast
this as the problem of generating images that combine the appearance of the
object as seen in a first example image with the geometry of the object as seen
in a second example image, where the two examples differ by a viewpoint change
and/or an object deformation. In order to factorize appearance and geometry, we
introduce a tight bottleneck in the geometry-extraction process that selects
and distils geometry-related features. Compared to standard image generation
problems, which often use generative adversarial networks, our generation task
is conditioned on both appearance and geometry and thus is significantly less
ambiguous, to the point that adopting a simple perceptual loss formulation is
sufficient. We demonstrate that our approach can learn object landmarks from
synthetic image deformations or videos, all without manual supervision, while
outperforming state-of-the-art unsupervised landmark detectors. We further show
that our method is applicable to a large variety of datasets - faces, people,
3D objects, and digits - without any modifications.

提出了一种无需手动监督来学习视觉对象（例如面部中的眼睛和鼻子）的地标探测器的方法，通过几何提取过程中引入的紧密瓶颈，结合外观和几何来生成图片，该方法适用于多种数据集，包括人脸、人物、3D 对象和数字，同时在无监督地标检测方面优于现有最先进的技术。