Advances in face swapping have enabled the automatic generation of highly
realistic faces. Yet face swaps are perceived differently than when looking at
real faces, with key differences in viewer behavior surrounding the eyes. Face
swapping algorithms generally place no emphasis on the eyes, relying on pixel
or feature matching losses that consider the entire face to guide the training
process. We further investigate viewer perception of face swaps, focusing our
analysis on the presence of an uncanny valley effect. We additionally propose a
novel loss equation for the training of face swapping models, leveraging a
pretrained gaze estimation network to directly improve representation of the
eyes. We confirm that viewed face swaps do elicit uncanny responses from
viewers. Our proposed improvements significant reduce viewing angle errors
between face swaps and their source material. Our method additionally reduces
the prevalence of the eyes as a deciding factor when viewers perform deepfake
detection tasks. Our findings have implications on face swapping for special
effects, as digital avatars, as privacy mechanisms, and more; negative
responses from users could limit effectiveness in said applications. Our gaze
improvements are a first step towards alleviating negative viewer perceptions
via a targeted approach.

面部交换研究中，我们探讨了观察者对面部交换的感知，特别关注了反常谷效应的存在。我们提出了一种新的训练模型方法，通过利用预先训练的凝视估计网络来直接改善对眼睛的表示。研究结果表明，面部交换会引起观察者的反常反应，但通过我们的方法可以显著减少面部交换与源材料之间的视角误差，并减少眼睛在深假检测任务中的重要性。这些发现对于特效制作、数字化形象、隐私保护等领域具有重要意义，用户的负面反应可能限制这些应用的有效性。通过定向方法改善观察者的感知是一个重要的起点。