The SOTA face swap models still suffer the problem of either target identity
(i.e., shape) being leaked or the target non-identity attributes (i.e.,
background, hair) failing to be fully preserved in the final results. We show
that this insufficient disentanglement is caused by two flawed designs that
were commonly adopted in prior models: (1) counting on only one compressed
encoder to represent both the semantic-level non-identity facial
attributes(i.e., pose) and the pixel-level non-facial region details, which is
contradictory to satisfy at the same time; (2) highly relying on long
skip-connections between the encoder and the final generator, leaking a certain
amount of target face identity into the result. To fix them, we introduce a new
face swap framework called 'WSC-swap' that gets rid of skip connections and
uses two target encoders to respectively capture the pixel-level non-facial
region attributes and the semantic non-identity attributes in the face region.
To further reinforce the disentanglement learning for the target encoder, we
employ both identity removal loss via adversarial training (i.e., GAN) and the
non-identity preservation loss via prior 3DMM models like [11]. Extensive
experiments on both FaceForensics++ and CelebA-HQ show that our results
significantly outperform previous works on a rich set of metrics, including one
novel metric for measuring identity consistency that was completely neglected
before.

提出了一种名为 'WSC-swap' 的新的面部交换框架，通过消除 skip connections 并利用两个目标编码器来分别捕获非脸部区域属性和脸部区域的语义非身份属性，并使用对抗训练中的身份移除损失和 3DMM 模型中的非身份保留损失来进一步加强目标编码器的解开交织性，并在 FaceForensics ++ 和 CelebA-HQ 上进行了广泛的实验。