Due to the omnipresence of Neural Radiance Fields (NeRFs), the interest
towards editable implicit 3D representations has surged over the last years.
However, editing implicit or hybrid representations as used for NeRFs is
difficult due to the entanglement of appearance and geometry encoded in the
model parameters. Despite these challenges, recent research has shown first
promising steps towards photorealistic and non-photorealistic appearance edits.
The main open issues of related work include limited interactivity, a lack of
support for local edits and large memory requirements, rendering them less
useful in practice. We address these limitations with LAENeRF, a unified
framework for photorealistic and non-photorealistic appearance editing of
NeRFs. To tackle local editing, we leverage a voxel grid as starting point for
region selection. We learn a mapping from expected ray terminations to final
output color, which can optionally be supervised by a style loss, resulting in
a framework which can perform photorealistic and non-photorealistic appearance
editing of selected regions. Relying on a single point per ray for our mapping,
we limit memory requirements and enable fast optimization. To guarantee
interactivity, we compose the output color using a set of learned, modifiable
base colors, composed with additive layer mixing. Compared to concurrent work,
LAENeRF enables recoloring and stylization while keeping processing time low.
Furthermore, we demonstrate that our approach surpasses baseline methods both
quantitatively and qualitatively.

用于编辑 NeRFs 的 LAENeRF 是一个统一框架，以实现对 NeRFs 的逼真和非逼真的外观编辑，采用区域选择的体素网格作为局部编辑的起点，并通过学习从射线末端到最终输出颜色的映射来限制内存需求和实现快速优化，最终实现了颜色重新着色和样式化，并在定量和定性方面超越了基线方法。

LAENeRF：神经辐射场的局部外观编辑

LAENeRF: Local Appearance Editing for Neural Radiance Fields

Natural language offers a highly intuitive interface for image editing. In
this paper, we introduce the first solution for performing local (region-based)
edits in generic natural images, based on a natural language description along
with an ROI mask. We achieve our goal by leveraging and combining a pretrained
language-image model (CLIP), to steer the edit towards a user-provided text
prompt, with a denoising diffusion probabilistic model (DDPM) to generate
natural-looking results. To seamlessly fuse the edited region with the
unchanged parts of the image, we spatially blend noised versions of the input
image with the local text-guided diffusion latent at a progression of noise
levels. In addition, we show that adding augmentations to the diffusion process
mitigates adversarial results. We compare against several baselines and related
methods, both qualitatively and quantitatively, and show that our method
outperforms these solutions in terms of overall realism, ability to preserve
the background and matching the text. Finally, we show several text-driven
editing applications, including adding a new object to an image,
removing/replacing/altering existing objects, background replacement, and image
extrapolation. Code is available at:
this https URL

本文介绍了一种基于自然语言描述和兴趣区域掩码进行本地（面向区域的）在通用自然图像中进行编辑的解决方案，利用 CLIP 预训练语言 - 图像模型指导编辑并使用扩散概率模型生成自然外观结果，以及应用扩散过程的增强进行抗击对抗攻击；通过在多个基线和相关方法上的定量和定性比较，表明该方法在整体逼真度、保护背景和匹配文本方面均优于现有解决方案，并展示了多种文本驱动的编辑应用，如添加新对象，删除 / 替换 / 修改现有对象，背景替换和图像推理。