3D generation has witnessed significant advancements, yet efficiently
producing high-quality 3D assets from a single image remains challenging. In
this paper, we present a triplane autoencoder, which encodes 3D models into a
compact triplane latent space to effectively compress both the 3D geometry and
texture information. Within the autoencoder framework, we introduce a 3D-aware
cross-attention mechanism, which utilizes low-resolution latent representations
to query features from a high-resolution 3D feature volume, thereby enhancing
the representation capacity of the latent space. Subsequently, we train a
diffusion model on this refined latent space. In contrast to solely relying on
image embedding for 3D generation, our proposed method advocates for the
simultaneous utilization of both image embedding and shape embedding as
conditions. Specifically, the shape embedding is estimated via a diffusion
prior model conditioned on the image embedding. Through comprehensive
experiments, we demonstrate that our method outperforms state-of-the-art
algorithms, achieving superior performance while requiring less training data
and time. Our approach enables the generation of high-quality 3D assets in
merely 7 seconds on a single A100 GPU.

通过使用 triplane 自编码器将 3D 模型编码为紧凑的 triplane 潜空间，本文提出了一种有效压缩 3D 几何和纹理信息的方法，并引入了 3D 感知交叉注意机制，从高分辨率的 3D 特征体积中查询特征，提高了潜空间的表示能力。同时利用图像嵌入和形状嵌入作为条件，通过扩散先验模型估计形状嵌入，实现了优于现有算法的性能，且仅在单个 A100 GPU 上耗时 7 秒。