Single-image relighting is a challenging task that involves reasoning about
the complex interplay between geometry, materials, and lighting. Many prior
methods either support only specific categories of images, such as portraits,
or require special capture conditions, like using a flashlight. Alternatively,
some methods explicitly decompose a scene into intrinsic components, such as
normals and BRDFs, which can be inaccurate or under-expressive. In this work,
we propose a novel end-to-end 2D relighting diffusion model, called Neural
Gaffer, that takes a single image of any object and can synthesize an accurate,
high-quality relit image under any novel environmental lighting condition,
simply by conditioning an image generator on a target environment map, without
an explicit scene decomposition. Our method builds on a pre-trained diffusion
model, and fine-tunes it on a synthetic relighting dataset, revealing and
harnessing the inherent understanding of lighting present in the diffusion
model. We evaluate our model on both synthetic and in-the-wild Internet imagery
and demonstrate its advantages in terms of generalization and accuracy.
Moreover, by combining with other generative methods, our model enables many
downstream 2D tasks, such as text-based relighting and object insertion. Our
model can also operate as a strong relighting prior for 3D tasks, such as
relighting a radiance field.

提出了一种新颖的端到端二维照明扩散模型 Neural Gaffer，通过将图像生成器条件化于目标环境图，可以在任意新环境照明条件下综合出准确、高质量的复照图像，而不需要显式场景分解。