We introduce VIVE3D, a novel approach that extends the capabilities of
image-based 3D GANs to video editing and is able to represent the input video
in an identity-preserving and temporally consistent way. We propose two new
building blocks. First, we introduce a novel GAN inversion technique
specifically tailored to 3D GANs by jointly embedding multiple frames and
optimizing for the camera parameters. Second, besides traditional semantic face
edits (e.g. for age and expression), we are the first to demonstrate edits that
show novel views of the head enabled by the inherent properties of 3D GANs and
our optical flow-guided compositing technique to combine the head with the
background video. Our experiments demonstrate that VIVE3D generates
high-fidelity face edits at consistent quality from a range of camera
viewpoints which are composited with the original video in a temporally and
spatially consistent manner.

通过提出 VIVE3D 方法，我们引入了一种新的方法，扩展了基于图像的三维生成对抗网络的能力，使其能够在保持身份不变和保持时间一致的同时呈现输入视频。我们提出了两个新的构建块。 第一个是针对三维生成对抗网络专门定制的新颖 GAN 反演技术，通过联合嵌入多个帧并优化相机参数来实现。其次，除了传统的语义面部编辑（例如年龄和表情），我们首次展示了借助 3D GANs 的固有属性和我们的光流引导合成技术显示头部的新视图的编辑，并将其与背景视频结合。我们的实验表明，VIVE3D 以一致的质量从一系列摄像机视角生成高保真度的面部编辑，并以时间和空间一致的方式与原始视频合成。