Current Neural Radiance Fields (NeRF) can generate photorealistic novel views. For editing 3D scenes represented by NeRF, with the advent of generative models, this paper proposes Inpaint4DNeRF to capitalize on state-of-the-art stable diffusion models (e.g., ControlNet) for direct generation of the underlying completed background content, regardless of static or dynamic. The key advantages of this generative approach for NeRF inpainting are twofold. First, after rough mask propagation, to complete or fill in previously occluded content, we can individually generate a small subset of completed images with plausible content, called seed images, from which simple 3D geometry proxies can be derived. Second and the remaining problem is thus 3D multiview consistency among all completed images, now guided by the seed images and their 3D proxies. Without other bells and whistles, our generative Inpaint4DNeRF baseline framework is general which can be readily extended to 4D dynamic NeRFs, where temporal consistency can be naturally handled in a similar way as our multiview consistency.

利用生成模型Inpaint4DNeRF，该论文提出了一种基于稳定扩散模型（如ControlNet）的直接生成方法，在编辑由NeRF表示的三维场景时，生成底层完成的背景内容，不论是静态还是动态，以实现NeRF修复的双重优势：通过生成种子图像和它们的三维代理，我们可以分别生成一小部分包含合理内容的完成图像，然后在所有完成图像中进行三维多视图一致性维护。

Inpaint4DNeRF: 具有生成扩散模型的可提示的时空 NeRF 补全