We propose NeRF-Insert, a NeRF editing framework that allows users to make high-quality local edits with a flexible level of control. Unlike previous work that relied on image-to-image models, we cast scene editing as an in-painting problem, which encourages the global structure of the scene to be preserved. Moreover, while most existing methods use only textual prompts to condition edits, our framework accepts a combination of inputs of different modalities as reference. More precisely, a user may provide a combination of textual and visual inputs including images, CAD models, and binary image masks for specifying a 3D region. We use generic image generation models to in-paint the scene from multiple viewpoints, and lift the local edits to a 3D-consistent NeRF edit. Compared to previous methods, our results show better visual quality and also maintain stronger consistency with the original NeRF.

我们提出了一种NeRF编辑框架NeRF-Insert，允许用户以灵活的控制水平进行高质量的本地编辑。与先前依赖于图像到图像模型的工作不同，我们将场景编辑视为一种修补问题，鼓励保持全局结构的场景。此外，尽管大多数现有方法仅使用文本提示来调整编辑，我们的框架接受不同模态的输入组合作为参考，包括图像、CAD模型和二进制图像掩码来指定一个3D区域。我们使用通用图像生成模型从多个视点修复场景，并将本地编辑提升为与原始NeRF保持一致的3D编辑。与先前方法相比，我们的结果表现出更好的视觉质量，并与原始NeRF保持更强的一致性。

NeRF-Insert：基于多模态控制信号的三维局部编辑