We present Boosting3D, a multi-stage single image-to-3D generation method
that can robustly generate reasonable 3D objects in different data domains. The
point of this work is to solve the view consistency problem in single
image-guided 3D generation by modeling a reasonable geometric structure. For
this purpose, we propose to utilize better 3D prior to training the NeRF. More
specifically, we train an object-level LoRA for the target object using
original image and the rendering output of NeRF. And then we train the LoRA and
NeRF using a progressive training strategy. The LoRA and NeRF will boost each
other while training. After the progressive training, the LoRA learns the 3D
information of the generated object and eventually turns to an object-level 3D
prior. In the final stage, we extract the mesh from the trained NeRF and use
the trained LoRA to optimize the structure and appearance of the mesh. The
experiments demonstrate the effectiveness of the proposed method. Boosting3D
learns object-specific 3D prior which is beyond the ability of pre-trained
diffusion priors and achieves state-of-the-art performance in the single
image-to-3d generation task.

Boosting3D 是一种多阶段单图像到 3D 生成方法，能够在不同数据领域中鲁棒地生成合理的 3D 对象。该方法通过建模合理的几何结构来解决单图像引导的 3D 生成中的视角一致性问题，利用更好的 3D 先验在训练 NeRF 之前对目标对象进行训练，并提出了一种渐进训练策略来训练 LoRA 和 NeRF。通过渐进训练，LoRA 学习生成对象的 3D 信息，最终成为对象级别的 3D 先验。实验证明了该方法的有效性，Boosting3D 学习了超出预训练扩散先验能力的对象特定 3D 先验，并在单图像到 3D 生成任务中取得了最先进的性能。