Improvements in data-capture and face modeling techniques have enabled us to
create high-fidelity realistic face models. However, driving these realistic
face models requires special input data, e.g. 3D meshes and unwrapped textures.
Also, these face models expect clean input data taken under controlled lab
environments, which is very different from data collected in the wild. All
these constraints make it challenging to use the high-fidelity models in
tracking for commodity cameras. In this paper, we propose a self-supervised
domain adaptation approach to enable the animation of high-fidelity face models
from a commodity camera. Our approach first circumvents the requirement for
special input data by training a new network that can directly drive a face
model just from a single 2D image. Then, we overcome the domain mismatch
between lab and uncontrolled environments by performing self-supervised domain
adaptation based on "consecutive frame texture consistency" based on the
assumption that the appearance of the face is consistent over consecutive
frames, avoiding the necessity of modeling the new environment such as lighting
or background. Experiments show that we are able to drive a high-fidelity face
model to perform complex facial motion from a cellphone camera without
requiring any labeled data from the new domain.

本文提出了一个用于自然环境下高保真面部模型的自我监督方法。该方法通过训练一个新的神经网络，使其直接从单个 2D 图像驱动面部模型，然后基于连续帧纹理的一致性进行自监督领域自适应，以在不需要建模新环境的情况下，克服了实验室和无控制环境之间的域不匹配问题。实验表明，我们能够在手机相机下驱动一个高保真面部模型，而不需要来自新领域的任何标记数据。