Recent advances in 2D image generation have achieved remarkable quality,largely driven by the capacity of diffusion models and the availability of large-scale datasets. However, direct 3D generation is still constrained by the scarcity and lower fidelity of 3D datasets. In this paper, we introduce Zero-1-to-G, a novel approach that addresses this problem by enabling direct single-view generation on Gaussian splats using pretrained 2D diffusion models. Our key insight is that Gaussian splats, a 3D representation, can be decomposed into multi-view images encoding different attributes. This reframes the challenging task of direct 3D generation within a 2D diffusion framework, allowing us to leverage the rich priors of pretrained 2D diffusion models. To incorporate 3D awareness, we introduce cross-view and cross-attribute attention layers, which capture complex correlations and enforce 3D consistency across generated splats. This makes Zero-1-to-G the first direct image-to-3D generative model to effectively utilize pretrained 2D diffusion priors, enabling efficient training and improved generalization to unseen objects. Extensive experiments on both synthetic and in-the-wild datasets demonstrate superior performance in 3D object generation, offering a new approach to high-quality 3D generation.

本研究解决了直接三维生成因三维数据集稀缺和质量低下而面临的挑战。提出的Zero-1-to-G方法通过引入Gaussian splats，使得预训练的二维扩散模型得以直接生成单视图，利用多视图图像来编码不同属性，并通过交叉视图和交叉属性注意力层增强三维一致性。实验表明，该方法在三维对象生成中表现出色，显著提高了生成质量和对新对象的泛化能力。

零-1到-G：驯服预训练的二维扩散模型以实现直接的三维生成