Facial video inpainting plays a crucial role in a wide range of applications,
including but not limited to the removal of obstructions in video conferencing
and telemedicine, enhancement of facial expression analysis, privacy
protection, integration of graphical overlays, and virtual makeup. This domain
presents serious challenges due to the intricate nature of facial features and
the inherent human familiarity with faces, heightening the need for accurate
and persuasive completions. In addressing challenges specifically related to
occlusion removal in this context, our focus is on the progressive task of
generating complete images from facial data covered by masks, ensuring both
spatial and temporal coherence. Our study introduces a network designed for
expression-based video inpainting, employing generative adversarial networks
(GANs) to handle static and moving occlusions across all frames. By utilizing
facial landmarks and an occlusion-free reference image, our model maintains the
user's identity consistently across frames. We further enhance emotional
preservation through a customized facial expression recognition (FER) loss
function, ensuring detailed inpainted outputs. Our proposed framework exhibits
proficiency in eliminating occlusions from facial videos in an adaptive form,
whether appearing static or dynamic on the frames, while providing realistic
and coherent results.

本研究中我们提出了一种用于表情驱动的视频修复的网络，利用生成对抗网络来处理静态和移动的遮挡物以生成完整的人脸图像，并借助面部标志和无遮挡参考图像来保持用户身份的一致性和情感的细节修复，为静态和动态帧中的面部视频消除遮挡提供了逼真和连贯的结果。