Denoising diffusion models have enabled high-quality image generation and
editing. We present a method to localize the desired edit region implicit in a
text instruction. We leverage InstructPix2Pix (IP2P) and identify the
discrepancy between IP2P predictions with and without the instruction. This
discrepancy is referred to as the relevance map. The relevance map conveys the
importance of changing each pixel to achieve the edits, and is used to to guide
the modifications. This guidance ensures that the irrelevant pixels remain
unchanged. Relevance maps are further used to enhance the quality of
text-guided editing of 3D scenes in the form of neural radiance fields. A field
is trained on relevance maps of training views, denoted as the relevance field,
defining the 3D region within which modifications should be made. We perform
iterative updates on the training views guided by rendered relevance maps from
the relevance field. Our method achieves state-of-the-art performance on both
image and NeRF editing tasks. Project page:
this https URL

我们提出了一种方法来定位文本指令中所隐含的期望编辑区域，并利用 InstructPix2Pix（IP2P）来确定在有和无指令的情况下 IP2P 预测之间的差异，以此差异作为相关性地图。相关性地图传达了每个像素更改的重要性，并用于指导修改，以确保不相关的像素保持不变。相关性地图进一步用于增强以神经辐射场形式的文本引导下 3D 场景的编辑质量。我们的方法在图像和 NeRF 编辑任务上实现了最先进的性能。