Text-guided image editing can have a transformative impact in supporting
creative applications. A key challenge is to generate edits that are faithful
to input text prompts, while consistent with input images. We present Imagen
Editor, a cascaded diffusion model built, by fine-tuning Imagen on text-guided
image inpainting. Imagen Editor's edits are faithful to the text prompts, which
is accomplished by using object detectors to propose inpainting masks during
training. In addition, Imagen Editor captures fine details in the input image
by conditioning the cascaded pipeline on the original high resolution image. To
improve qualitative and quantitative evaluation, we introduce EditBench, a
systematic benchmark for text-guided image inpainting. EditBench evaluates
inpainting edits on natural and generated images exploring objects, attributes,
and scenes. Through extensive human evaluation on EditBench, we find that
object-masking during training leads to across-the-board improvements in
text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and
Stable Diffusion -- and, as a cohort, these models are better at
object-rendering than text-rendering, and handle material/color/size attributes
better than count/shape attributes.

本文提出了一种基于文本指导的图像编辑系统 Imagen Editor，通过利用对象检测器提出文本提示的修复遮罩并使用高分辨率图像来处理细节来实现忠实于输入文本且与输入图像一致的编辑，并提出一个名为 EditBench 的系统化基准来评估编辑效果。研究发现在训练过程中进行对象遮罩处理，可以显著提高文本与图像的对齐度和整体渲染效果，同时相较于文本渲染而言，这种方法在物体渲染和处理材料 / 颜色 / 大小属性上效果更佳。