We propose a novel recurrent encoder-decoder network model for real-time
video-based face alignment. Our proposed model predicts 2D facial point maps
regularized by a regression loss, while uniquely exploiting recurrent learning
at both spatial and temporal dimensions. At the spatial level, we add a
feedback loop connection between the combined output response map and the
input, in order to enable iterative coarse-to-fine face alignment using a
single network model. At the temporal level, we first decouple the features in
the bottleneck of the network into temporal-variant factors, such as pose and
expression, and temporal-invariant factors, such as identity information.
Temporal recurrent learning is then applied to the decoupled temporal-variant
features, yielding better generalization and significantly more accurate
results at test time. We perform a comprehensive experimental analysis, showing
the importance of each component of our proposed model, as well as superior
results over the state-of-the-art in standard datasets.

本研究提出了一种新颖的循环编码器 - 解码器网络模型，用于实时基于视频的面部对齐。我们的模型预测带有回归损失的 2D 面部关键点映射，同时在空间和时间维度上独特地利用循环学习。在空间层面上，我们在联合输出响应映射和输入之间添加反馈循环连接，以使用单个网络模型实现迭代式的粗到精面部对齐。在时间层面上，我们首先将网络瓶颈中的特征分解为时间可变因素和时间不变因素，例如姿势和表情以及身份信息。然后将时间循环学习应用于分解的时间可变特征，在测试时间产生更好的普适性和显着更精确的结果。通过全面实验分析，我们展示了提出的模型各组件的重要性，以及在标准数据集上的优异结果。