We present a two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors. The first stage samples from a 3D diffusion prior directly learned from 3D data. Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping. The second stage utilizes 2D diffusion priors to further refine the texture of coarse 3D models from the first stage. The refinement consists of both latent and pixel space optimization for high-quality texture generation. To facilitate the training of the proposed system, we clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models. Experiment results are reported qualitatively and quantitatively to show the performance of the proposed system. Our codes and models are available at https://github.com/3DTopia/3DTopia

我们提出了一个两阶段的文本到3D生成系统，名为3DTopia，它使用混合扩散先验在5分钟内生成高质量的通用3D资产。第一阶段直接从3D数据学习的3D扩散先验中采样，由文本条件的三视图潜在扩散模型提供动力，快速生成用于快速原型设计的粗糙3D样本。第二阶段利用2D扩散先验进一步优化第一阶段中粗糙3D模型的纹理。优化包括潜在空间和像素空间的优化，用于生成高质量的纹理。为了促进所提出系统的训练，我们通过结合视觉语言模型和大型语言模型清洗和加标最大的开源3D数据集Objaverse。实验证据定性和定量地展示了所提出系统的性能。我们的代码和模型可在此https网址获得。

3DTopia: 带混合扩散先验的大规模文本到3D生成模型