In this work, we propose a novel two-stage framework, called FaceShifter, for
high fidelity and occlusion aware face swapping. Unlike many existing face
swapping works that leverage only limited information from the target image
when synthesizing the swapped face, our framework, in its first stage,
generates the swapped face in high-fidelity by exploiting and integrating the
target attributes thoroughly and adaptively. We propose a novel attributes
encoder for extracting multi-level target face attributes, and a new generator
with carefully designed Adaptive Attentional Denormalization (AAD) layers to
adaptively integrate the identity and the attributes for face synthesis. To
address the challenging facial occlusions, we append a second stage consisting
of a novel Heuristic Error Acknowledging Refinement Network (HEAR-Net). It is
trained to recover anomaly regions in a self-supervised way without any manual
annotations. Extensive experiments on wild faces demonstrate that our face
swapping results are not only considerably more perceptually appealing, but
also better identity preserving in comparison to other state-of-the-art
methods.

本研究提出了一种名为 FaceShifter 的新型两阶段框架，用于高保真度和遮挡感知的人脸换脸。该框架通过完全自适应地利用和整合目标属性，在第一阶段生成高清晰度的换脸图像，其创新点在于引入目标属性编码器和自适应关注规范化层生成器。为了解决复杂的面部遮挡问题，我们还增加了由 Heuristic Error Acknowledging Refinement Network (HEAR-Net) 组成的第二阶段。大量实验表明，我们的技术不仅在感知上更好，而且还有更好的身份保护能力。