We present Face2Face, a novel approach for real-time facial reenactment of a
monocular target video sequence (e.g., Youtube video). The source sequence is
also a monocular video stream, captured live with a commodity webcam. Our goal
is to animate the facial expressions of the target video by a source actor and
re-render the manipulated output video in a photo-realistic fashion. To this
end, we first address the under-constrained problem of facial identity recovery
from monocular video by non-rigid model-based bundling. At run time, we track
facial expressions of both source and target video using a dense photometric
consistency measure. Reenactment is then achieved by fast and efficient
deformation transfer between source and target. The mouth interior that best
matches the re-targeted expression is retrieved from the target sequence and
warped to produce an accurate fit. Finally, we convincingly re-render the
synthesized target face on top of the corresponding video stream such that it
seamlessly blends with the real-world illumination. We demonstrate our method
in a live setup, where Youtube videos are reenacted in real time.

Face2Face 是一种用于实时面部复刻的创新方法，可以通过非刚性基于模型的束缚，跟踪面部表情，实现来自单眼录制的目标视频序列的面部表情的动画化，并以逼真的方式重新渲染操作后的输出视频。

Face2Face: 实时 RGB 视频人脸捕获和再现

Face2Face: Real-time Face Capture and Reenactment of RGB Videos

We propose a novel learnable representation for detail-preserving shape
deformation. The goal of our method is to warp a source shape to match the
general structure of a target shape, while preserving the surface details of
the source. Our method extends a traditional cage-based deformation technique,
where the source shape is enclosed by a coarse control mesh termed \emph{cage},
and translations prescribed on the cage vertices are interpolated to any point
on the source mesh via special weight functions. The use of this sparse cage
scaffolding enables preserving surface details regardless of the shape's
intricacy and topology. Our key contribution is a novel neural network
architecture for predicting deformations by controlling the cage. We
incorporate a differentiable cage-based deformation module in our architecture,
and train our network end-to-end. Our method can be trained with common
collections of 3D models in an unsupervised fashion, without any cage-specific
annotations. We demonstrate the utility of our method for synthesizing shape
variations and deformation transfer.

本文提出了一种可学习的表示方法，用于形变中保留细节，其核心是一种基于 cage 的形变技术并且结合了神经网络来控制 cage 的变形以保留细节，可在无监督学习的情况下进行训练，并且在形状变化和变形传输方面具有实用价值。