Text-to-3D generation, which aims to synthesize vivid 3D objects from text
prompts, has attracted much attention from the computer vision community. While
several existing works have achieved impressive results for this task, they
mainly rely on a time-consuming optimization paradigm. Specifically, these
methods optimize a neural field from scratch for each text prompt, taking
approximately one hour or more to generate one object. This heavy and
repetitive training cost impedes their practical deployment. In this paper, we
propose a novel framework for fast text-to-3D generation, dubbed Instant3D.
Once trained, Instant3D is able to create a 3D object for an unseen text prompt
in less than one second with a single run of a feedforward network. We achieve
this remarkable speed by devising a new network that directly constructs a 3D
triplane from a text prompt. The core innovation of our Instant3D lies in our
exploration of strategies to effectively inject text conditions into the
network. Furthermore, we propose a simple yet effective activation function,
the scaled-sigmoid, to replace the original sigmoid function, which speeds up
the training convergence by more than ten times. Finally, to address the Janus
(multi-head) problem in 3D generation, we propose an adaptive Perp-Neg
algorithm that can dynamically adjust its concept negation scales according to
the severity of the Janus problem during training, effectively reducing the
multi-head effect. Extensive experiments on a wide variety of benchmark
datasets demonstrate that the proposed algorithm performs favorably against the
state-of-the-art methods both qualitatively and quantitatively, while achieving
significantly better efficiency. The project page is at
this https URL

提出了一种新的基于文本的 3D 生成框架，名为 Instant3D，使用前向网络在不到一秒的时间内为未见的文本提示创建一个 3D 对象，并解决了 Janus 问题的多头效应。

Instant3D：即时文本到三维生成

Instant3D: Instant Text-to-3D Generation

Text-to-3D with diffusion models have achieved remarkable progress in recent
years. However, existing methods either rely on score distillation-based
optimization which suffer from slow inference, low diversity and Janus
problems, or are feed-forward methods that generate low quality results due to
the scarcity of 3D training data. In this paper, we propose Instant3D, a novel
method that generates high-quality and diverse 3D assets from text prompts in a
feed-forward manner. We adopt a two-stage paradigm, which first generates a
sparse set of four structured and consistent views from text in one shot with a
fine-tuned 2D text-to-image diffusion model, and then directly regresses the
NeRF from the generated images with a novel transformer-based sparse-view
reconstructor. Through extensive experiments, we demonstrate that our method
can generate high-quality, diverse and Janus-free 3D assets within 20 seconds,
which is two order of magnitude faster than previous optimization-based methods
that can take 1 to 10 hours. Our project webpage: this https URL

通过使用两阶段的方法 —— 首先使用精调的 2D 文本到图像扩散模型一次性生成稀疏一致的四个结构化视图，然后利用一种新颖的基于 Transformer 的稀疏视图重构器直接回归生成的图像的 NeRF—— 我们提出了 Instant3D，一种新颖的方法，以前馈方式从文本提示中生成高质量、多样化的 3D 资产。通过大量实验证明，我们的方法可以在 20 秒内生成高质量、多样化且无 Janus 问题的 3D 资产，比之前需要 1 到 10 小时的基于优化的方法快两个数量级。