Face swapping combines one face's identity with another face's non-appearance
attributes (expression, head pose, lighting) to generate a synthetic face. This
technology is rapidly improving, but falls flat when reconstructing some
attributes, particularly gaze. Image-based loss metrics that consider the full
face do not effectively capture the perceptually important, yet spatially
small, eye regions. Improving gaze in face swaps can improve naturalness and
realism, benefiting applications in entertainment, human computer interaction,
and more. Improved gaze will also directly improve Deepfake detection efforts,
serving as ideal training data for classifiers that rely on gaze for
classification. We propose a novel loss function that leverages gaze prediction
to inform the face swap model during training and compare against existing
methods. We find all methods to significantly benefit gaze in resulting face
swaps.

本研究提出了一种利用凝视识别辅助脸部交换模型训练的新型损失函数，以改善脸部交换过程中的目光朝向问题，实现更自然、逼真的图片生成。结果发现，该方法在改善凝视问题方面表现明显优于现有技术。