In this paper, we examine 3 important issues in the practical use of
state-of-the-art facial landmark detectors and show how a combination of
specific architectural modifications can directly improve their accuracy and
temporal stability. First, many facial landmark detectors require face
normalization as a preprocessing step, which is accomplished by a
separately-trained neural network that crops and resizes the face in the input
image. There is no guarantee that this pre-trained network performs the optimal
face normalization for landmark detection. We instead analyze the use of a
spatial transformer network that is trained alongside the landmark detector in
an unsupervised manner, and jointly learn optimal face normalization and
landmark detection. Second, we show that modifying the output head of the
landmark predictor to infer landmarks in a canonical 3D space can further
improve accuracy. To convert the predicted 3D landmarks into screen-space, we
additionally predict the camera intrinsics and head pose from the input image.
As a side benefit, this allows to predict the 3D face shape from a given image
only using 2D landmarks as supervision, which is useful in determining landmark
visibility among other things. Finally, when training a landmark detector on
multiple datasets at the same time, annotation inconsistencies across datasets
forces the network to produce a suboptimal average. We propose to add a
semantic correction network to address this issue. This additional lightweight
neural network is trained alongside the landmark detector, without requiring
any additional supervision. While the insights of this paper can be applied to
most common landmark detectors, we specifically target a recently-proposed
continuous 2D landmark detector to demonstrate how each of our additions leads
to meaningful improvements over the state-of-the-art on standard benchmarks.

这篇论文主要探讨了人脸标记探测器的实际应用中存在的三个重要问题，并展示了如何通过特定的架构改进来提高其准确性和时间稳定性。

无限 3D 关键点：提升连续 2D 面部关键点检测

Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detection

An efficient, fully automatic method for 3D face shape and pose estimation in
unconstrained 2D imagery is presented. The proposed method jointly estimates a
dense set of 3D landmarks and facial geometry using a single pass of a modified
version of the popular "U-Net" neural network architecture. Additionally, we
propose a method for directly estimating a set of 3D Morphable Model (3DMM)
parameters, using the estimated 3D landmarks and geometry as constraints in a
simple linear system. Qualitative modeling results are presented, as well as
quantitative evaluation of predicted 3D face landmarks in unconstrained video
sequences.

这篇论文介绍了一种高效、全自动的方法，可以在自由 2D 图像中进行 3D 人脸形状和姿态估计，并使用基于改进的 U-Net 神经网络架构的单一传递联合估计稠密的 3D 关键点和面部几何，此外，还提出一种直接估计 3D 可塑模型（3DMM）参数的方法，通过估计的 3D 关键点和几何形状作为线性系统的约束，为非约束性视频序列中的预测 3D 面部关键点进行了定性建模结果和定量评估。