Generating diverse and high-quality 3D assets automatically poses a
fundamental yet challenging task in 3D computer vision. Despite extensive
efforts in 3D generation, existing optimization-based approaches struggle to
produce large-scale 3D assets efficiently. Meanwhile, feed-forward methods
often focus on generating only a single category or a few categories, limiting
their generalizability. Therefore, we introduce a diffusion-based feed-forward
framework to address these challenges with a single model. To handle the large
diversity and complexity in geometry and texture across categories efficiently,
we 1) adopt improved triplane to guarantee efficiency; 2) introduce the
3D-aware transformer to aggregate the generalized 3D knowledge with specialized
3D features; and 3) devise the 3D-aware encoder/decoder to enhance the
generalized 3D knowledge. Building upon our 3D-aware Diffusion model with
TransFormer, DiffTF, we propose a stronger version for 3D generation, i.e.,
DiffTF++. It boils down to two parts: multi-view reconstruction loss and
triplane refinement. Specifically, we utilize multi-view reconstruction loss to
fine-tune the diffusion model and triplane decoder, thereby avoiding the
negative influence caused by reconstruction errors and improving texture
synthesis. By eliminating the mismatch between the two stages, the generative
performance is enhanced, especially in texture. Additionally, a 3D-aware
refinement process is introduced to filter out artifacts and refine triplanes,
resulting in the generation of more intricate and reasonable details. Extensive
experiments on ShapeNet and OmniObject3D convincingly demonstrate the
effectiveness of our proposed modules and the state-of-the-art 3D object
generation performance with large diversity, rich semantics, and high quality.

通过引入一种基于扩散的前馈框架，以及使用改进的三面平面和 3D-aware 转换器、编码器 / 解码器来处理大规模的 3D 资源生成任务，该研究提出了一个更强的 3D 生成模型，以增强多样性、语义和质量。