Virtual reality and augmented reality (XR) bring increasing demand for 3D content. However, creating high-quality 3D content requires tedious work that a human expert must do. In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360{\deg} views that correspond well with the given reference image. By conditioning on the reference image, our model can fulfill the everlasting curiosity for synthesizing novel views of objects from images. Our technique sheds light on a promising direction of easing the workflows for 3D artists and XR designers. We propose a novel framework, dubbed NeuralLift-360, that utilizes a depth-aware neural radiance representation (NeRF) and learns to craft the scene guided by denoising diffusion models. By introducing a ranking loss, our NeuralLift-360 can be guided with rough depth estimation in the wild. We also adopt a CLIP-guided sampling strategy for the diffusion prior to provide coherent guidance. Extensive experiments demonstrate that our NeuralLift-360 significantly outperforms existing state-of-the-art baselines. Project page: https://vita-group.github.io/NeuralLift-360/

本文提出一种名为NeuralLift-360的新框架，使用深度感知的神经辐射表示（NeRF）和去噪扩散模型学习场景，通过引入排名损失，可以使用自然图像进行粗略深度估计，这种技术可以缓解3D艺术家和XR设计师繁琐的工作流程，生成出高质量的360度3D对象，具有以下优点：自然图像合成、顶多的深度估计、一致的引导，大力打破现有基线模型！

NeuralLift-360: 将野外2D照片提升为带有360°观测的3D物体