Recent advances in generative AI have unveiled significant potential for the
creation of 3D content. However, current methods either apply a pre-trained 2D
diffusion model with the time-consuming score distillation sampling (SDS), or a
direct 3D diffusion model trained on limited 3D data losing generation
diversity. In this work, we approach the problem by employing a multi-view 2.5D
diffusion fine-tuned from a pre-trained 2D diffusion model. The multi-view 2.5D
diffusion directly models the structural distribution of 3D data, while still
maintaining the strong generalization ability of the original 2D diffusion
model, filling the gap between 2D diffusion-based and direct 3D diffusion-based
methods for 3D content generation. During inference, multi-view normal maps are
generated using the 2.5D diffusion, and a novel differentiable rasterization
scheme is introduced to fuse the almost consistent multi-view normal maps into
a consistent 3D model. We further design a normal-conditioned multi-view image
generation module for fast appearance generation given the 3D geometry. Our
method is a one-pass diffusion process and does not require any SDS
optimization as post-processing. We demonstrate through extensive experiments
that, our direct 2.5D generation with the specially-designed fusion scheme can
achieve diverse, mode-seeking-free, and high-fidelity 3D content generation in
only 10 seconds. Project page: this https URL

通过使用经过微调的 2D 扩散模型的多视角 2.5D 扩散方法，我们填补了 2D 扩散和直接 3D 扩散方法之间的差距，为 3D 内容生成提供了多样性、无模式寻找和高保真度，仅需 10 秒。