In this paper, we present supervision-by-registration, an unsupervised
approach to improve the precision of facial landmark detectors on both images
and video. Our key observation is that the detections of the same landmark in
adjacent frames should be coherent with registration, i.e., optical flow.
Interestingly, the coherency of optical flow is a source of supervision that
does not require manual labeling, and can be leveraged during detector
training. For example, we can enforce in the training loss function that a
detected landmark at frame$_{t-1}$ followed by optical flow tracking from
frame$_{t-1}$ to frame$_t$ should coincide with the location of the detection
at frame$_t$. Essentially, supervision-by-registration augments the training
loss function with a registration loss, thus training the detector to have
output that is not only close to the annotations in labeled images, but also
consistent with registration on large amounts of unlabeled videos. End-to-end
training with the registration loss is made possible by a differentiable
Lucas-Kanade operation, which computes optical flow registration in the forward
pass, and back-propagates gradients that encourage temporal coherency in the
detector. The output of our method is a more precise image-based facial
landmark detector, which can be applied to single images or video. With
supervision-by-registration, we demonstrate (1) improvements in facial landmark
detection on both images (300W, ALFW) and video (300VW, Youtube-Celebrities),
and (2) significant reduction of jittering in video detections.

通过图像和视频的无监督方法，利用光流法进行检测的统一性，提高面部标记检测器的精度，最终输出更精确的面部标记检测器，适用于单张图像或视频。