Text-to-3D with diffusion models have achieved remarkable progress in recent
years. However, existing methods either rely on score distillation-based
optimization which suffer from slow inference, low diversity and Janus
problems, or are feed-forward methods that generate low quality results due to
the scarcity of 3D training data. In this paper, we propose Instant3D, a novel
method that generates high-quality and diverse 3D assets from text prompts in a
feed-forward manner. We adopt a two-stage paradigm, which first generates a
sparse set of four structured and consistent views from text in one shot with a
fine-tuned 2D text-to-image diffusion model, and then directly regresses the
NeRF from the generated images with a novel transformer-based sparse-view
reconstructor. Through extensive experiments, we demonstrate that our method
can generate high-quality, diverse and Janus-free 3D assets within 20 seconds,
which is two order of magnitude faster than previous optimization-based methods
that can take 1 to 10 hours. Our project webpage: this https URL

通过使用两阶段的方法 —— 首先使用精调的 2D 文本到图像扩散模型一次性生成稀疏一致的四个结构化视图，然后利用一种新颖的基于 Transformer 的稀疏视图重构器直接回归生成的图像的 NeRF—— 我们提出了 Instant3D，一种新颖的方法，以前馈方式从文本提示中生成高质量、多样化的 3D 资产。通过大量实验证明，我们的方法可以在 20 秒内生成高质量、多样化且无 Janus 问题的 3D 资产，比之前需要 1 到 10 小时的基于优化的方法快两个数量级。