The success of image generative models has enabled us to build methods that
can edit images based on text or other user input. However, these methods are
bespoke, imprecise, require additional information, or are limited to only 2D
image edits. We present GeoDiffuser, a zero-shot optimization-based method that
unifies common 2D and 3D image-based object editing capabilities into a single
method. Our key insight is to view image editing operations as geometric
transformations. We show that these transformations can be directly
incorporated into the attention layers in diffusion models to implicitly
perform editing operations. Our training-free optimization method uses an
objective function that seeks to preserve object style but generate plausible
images, for instance with accurate lighting and shadows. It also inpaints
disoccluded parts of the image where the object was originally located. Given a
natural image and user input, we segment the foreground object using SAM and
estimate a corresponding transform which is used by our optimization approach
for editing. GeoDiffuser can perform common 2D and 3D edits like object
translation, 3D rotation, and removal. We present quantitative results,
including a perceptual study, that shows how our approach is better than
existing methods. Visit this https URL for
more information.

GeoDiffuser 是一种优化为基础的方法，将常见的二维和三维基于图像的对象编辑功能统一为单一方法，通过将图像编辑操作视为几何变换，并将其直接合并到扩散模型的注意力层中隐式执行编辑操作，以实现保持对象风格并生成合理图像的目标函数进行训练 - free 优化方法。