Neural reconstruction approaches are rapidly emerging as the preferred
representation for 3D scenes, but their limited editability is still posing a
challenge. In this work, we propose an approach for 3D scene inpainting -- the
task of coherently replacing parts of the reconstructed scene with desired
content. Scene inpainting is an inherently ill-posed task as there exist many
solutions that plausibly replace the missing content. A good inpainting method
should therefore not only enable high-quality synthesis but also a high degree
of control. Based on this observation, we focus on enabling explicit control
over the inpainted content and leverage a reference image as an efficient means
to achieve this goal. Specifically, we introduce RefFusion, a novel 3D
inpainting method based on a multi-scale personalization of an image inpainting
diffusion model to the given reference view. The personalization effectively
adapts the prior distribution to the target scene, resulting in a lower
variance of score distillation objective and hence significantly sharper
details. Our framework achieves state-of-the-art results for object removal
while maintaining high controllability. We further demonstrate the generality
of our formulation on other downstream tasks such as object insertion, scene
outpainting, and sparse view reconstruction.

我们提出了一种基于多尺度个性化图像修复扩散模型的新型三维修复方法 RefFusion，通过引入参考图像，实现了对修复内容的显式控制，从而在保持高可控性的同时取得了最先进的物体去除效果。