We present Face Swapping GAN (FSGAN) for face swapping and reenactment.
Unlike previous work, we offer a subject agnostic swapping scheme that can be
applied to pairs of faces without requiring training on those faces. We derive
a novel iterative deep learning--based approach for face reenactment which
adjusts significant pose and expression variations that can be applied to a
single image or a video sequence. For video sequences, we introduce a
continuous interpolation of the face views based on reenactment, Delaunay
Triangulation, and barycentric coordinates. Occluded face regions are handled
by a face completion network. Finally, we use a face blending network for
seamless blending of the two faces while preserving the target skin color and
lighting conditions. This network uses a novel Poisson blending loss combining
Poisson optimization with a perceptual loss. We compare our approach to
existing state-of-the-art systems and show our results to be both qualitatively
and quantitatively superior. This work describes extensions of the FSGAN
method, proposed in an earlier conference version of our work, as well as
additional experiments and results.

本研究提出了一种基于深度学习的面部交换生成对抗网络（FSGAN），用于面部交换和活体重放，包括一种新颖的迭代深度学习方法，可用于面部活体重放，采用重建三角形和重心坐标的方法连续插值，使用面部补全网络处理被遮挡的面部区域，最终采用一种新型泊松混合损失融合两张面孔并保持目标肤色和照明条件。

FSGANv2：改进的主体无关人脸交换和重现技术

FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment

Image composition is an important operation to create visual content. Among
image composition tasks, image blending aims to seamlessly blend an object from
a source image onto a target image with lightly mask adjustment. A popular
approach is Poisson image blending, which enforces the gradient domain
smoothness in the composite image. However, this approach only considers the
boundary pixels of target image, and thus can not adapt to texture of target
image. In addition, the colors of the target image often seep through the
original source object too much causing a significant loss of content of the
source object. We propose a Poisson blending loss that achieves the same
purpose of Poisson image blending. In addition, we jointly optimize the
proposed Poisson blending loss as well as the style and content loss computed
from a deep network, and reconstruct the blending region by iteratively
updating the pixels using the L-BFGS solver. In the blending image, we not only
smooth out gradient domain of the blending boundary but also add consistent
texture into the blending region. User studies show that our method outperforms
strong baselines as well as state-of-the-art approaches when placing objects
onto both paintings and real-world images.

提出了使用 Poisson 混合损失函数结合深度学习中的风格和内容损失函数，使用 L-BFGS 迭代求解器对图像进行优化，既满足了图像混合过渡自然性和良好的纹理一致性，又能更好的应用于真实图像和油画等情况。

深度图像融合

Deep Image Blending

We present Face Swapping GAN (FSGAN) for face swapping and reenactment.
Unlike previous work, FSGAN is subject agnostic and can be applied to pairs of
faces without requiring training on those faces. To this end, we describe a
number of technical contributions. We derive a novel recurrent neural network
(RNN)-based approach for face reenactment which adjusts for both pose and
expression variations and can be applied to a single image or a video sequence.
For video sequences, we introduce continuous interpolation of the face views
based on reenactment, Delaunay Triangulation, and barycentric coordinates.
Occluded face regions are handled by a face completion network. Finally, we use
a face blending network for seamless blending of the two faces while preserving
target skin color and lighting conditions. This network uses a novel Poisson
blending loss which combines Poisson optimization with perceptual loss. We
compare our approach to existing state-of-the-art systems and show our results
to be both qualitatively and quantitatively superior.

本文提出了一种基于 GAN 的脸部交换和演绎方法，其中采用了 RNN 和 Delaunay Triangulation 来进行脸部再现，并使用 Poisson blending loss 来实现融合。与现有技术相比，本方法在定性和定量方面均有更好的表现。