We present Supervision by Registration and Triangulation (SRT), an
unsupervised approach that utilizes unlabeled multi-view video to improve the
accuracy and precision of landmark detectors. Being able to utilize unlabeled
data enables our detectors to learn from massive amounts of unlabeled data
freely available and not be limited by the quality and quantity of manual human
annotations. To utilize unlabeled data, there are two key observations: (1) the
detections of the same landmark in adjacent frames should be coherent with
registration, i.e., optical flow. (2) the detections of the same landmark in
multiple synchronized and geometrically calibrated views should correspond to a
single 3D point, i.e., multi-view consistency. Registration and multi-view
consistency are sources of supervision that do not require manual labeling,
thus it can be leveraged to augment existing training data during detector
training. End-to-end training is made possible by differentiable registration
and 3D triangulation modules. Experiments with 11 datasets and a newly proposed
metric to measure precision demonstrate accuracy and precision improvements in
landmark detection on both images and video. Code is available at
this https URL

本文介绍了一种无人监督的方法，使用未标记的多视角视频来改善地标探测器的精度和准确性，利用不需要人工注释的监督源，通过可微的配准和三维三角测量模块进行端对端训练，实验表明该方法可以提高地标检测在图像和视频中的精度和准确性。