Existing single image-to-3D creation methods typically involve a two-stage process, first generating multi-view images, and then using these images for 3D reconstruction. However, training these two stages separately leads to significant data bias in the inference phase, thus affecting the quality of reconstructed results. We introduce a unified 3D generation framework, named Ouroboros3D, which integrates diffusion-based multi-view image generation and 3D reconstruction into a recursive diffusion process. In our framework, these two modules are jointly trained through a self-conditioning mechanism, allowing them to adapt to each other's characteristics for robust inference. During the multi-view denoising process, the multi-view diffusion model uses the 3D-aware maps rendered by the reconstruction module at the previous timestep as additional conditions. The recursive diffusion framework with 3D-aware feedback unites the entire process and improves geometric consistency.Experiments show that our framework outperforms separation of these two stages and existing methods that combine them at the inference phase. Project page: https://costwen.github.io/Ouroboros3D/

现有的单图像到3D生成方法通常涉及两个阶段的过程，首先生成多视图图像，然后使用这些图像进行3D重建。然而，分别训练这两个阶段会导致推理阶段的数据偏差，从而影响重建结果的质量。我们引入了一个统一的3D生成框架，命名为Ouroboros3D，它将基于扩散的多视图图像生成和3D重建集成到递归扩散过程中。在我们的框架中，通过自我条件机制联合训练这两个模块，使它们能够适应彼此的特征以进行稳健的推理。在多视图去噪过程中，多视图扩散模型使用由重建模块在上一时间步骤渲染的具有3D感知的地图作为附加条件。具有3D感知反馈的递归扩散框架统一了整个过程并改进了几何一致性。实验证明，我们的框架优于将这两个阶段分离和将它们合并在推理阶段的现有方法。

Ouroboros3D: 图像到三维生成的三维感知递归扩散