3D generation has raised great attention in recent years. With the success of
text-to-image diffusion models, the 2D-lifting technique becomes a promising
route to controllable 3D generation. However, these methods tend to present
inconsistent geometry, which is also known as the Janus problem. We observe
that the problem is caused mainly by two aspects, i.e., viewpoint bias in 2D
diffusion models and overfitting of the optimization objective. To address it,
we propose a two-stage 2D-lifting framework, namely DreamControl, which
optimizes coarse NeRF scenes as 3D self-prior and then generates fine-grained
objects with control-based score distillation. Specifically, adaptive viewpoint
sampling and boundary integrity metric are proposed to ensure the consistency
of generated priors. The priors are then regarded as input conditions to
maintain reasonable geometries, in which conditional LoRA and weighted score
are further proposed to optimize detailed textures. DreamControl can generate
high-quality 3D content in terms of both geometry consistency and texture
fidelity. Moreover, our control-based optimization guidance is applicable to
more downstream tasks, including user-guided generation and 3D animation. The
project page is available at this https URL

通过 DreamControl 的两阶段 2D-lifting 框架，实现了在 3D 生成中解决几何不一致性的 Janus 问题，并生成出几何一致性和纹理保真度都很高的高质量 3D 内容。

DreamControl: 基于控制的文本到 3D 生成与 3D 自先

DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior

Text-to-3D generation, which aims to synthesize vivid 3D objects from text
prompts, has attracted much attention from the computer vision community. While
several existing works have achieved impressive results for this task, they
mainly rely on a time-consuming optimization paradigm. Specifically, these
methods optimize a neural field from scratch for each text prompt, taking
approximately one hour or more to generate one object. This heavy and
repetitive training cost impedes their practical deployment. In this paper, we
propose a novel framework for fast text-to-3D generation, dubbed Instant3D.
Once trained, Instant3D is able to create a 3D object for an unseen text prompt
in less than one second with a single run of a feedforward network. We achieve
this remarkable speed by devising a new network that directly constructs a 3D
triplane from a text prompt. The core innovation of our Instant3D lies in our
exploration of strategies to effectively inject text conditions into the
network. Furthermore, we propose a simple yet effective activation function,
the scaled-sigmoid, to replace the original sigmoid function, which speeds up
the training convergence by more than ten times. Finally, to address the Janus
(multi-head) problem in 3D generation, we propose an adaptive Perp-Neg
algorithm that can dynamically adjust its concept negation scales according to
the severity of the Janus problem during training, effectively reducing the
multi-head effect. Extensive experiments on a wide variety of benchmark
datasets demonstrate that the proposed algorithm performs favorably against the
state-of-the-art methods both qualitatively and quantitatively, while achieving
significantly better efficiency. The project page is at
this https URL

提出了一种新的基于文本的 3D 生成框架，名为 Instant3D，使用前向网络在不到一秒的时间内为未见的文本提示创建一个 3D 对象，并解决了 Janus 问题的多头效应。

Instant3D：即时文本到三维生成

Instant3D: Instant Text-to-3D Generation

While the image diffusion model has made significant strides in text-driven
3D content creation, it often falls short in accurately capturing the intended
meaning of the text prompt, particularly with respect to direction information.
This shortcoming gives rise to the Janus problem, where multi-faced 3D models
are produced with the guidance of such diffusion models. In this paper, we
present a robust pipeline for generating high-fidelity 3D content with
orthogonal-view image guidance. Specifically, we introduce a novel 2D diffusion
model that generates an image consisting of four orthogonal-view sub-images for
the given text prompt. The 3D content is then created with this diffusion
model, which enhances 3D consistency and provides strong structured semantic
priors. This addresses the infamous Janus problem and significantly promotes
generation efficiency. Additionally, we employ a progressive 3D synthesis
strategy that results in substantial improvement in the quality of the created
3D contents. Both quantitative and qualitative evaluations show that our method
demonstrates a significant improvement over previous text-to-3D techniques.

通过引入一种新的二维扩散模型来生成由四个正交视图子图像组成的图像，从而利用正交视图图像引导生成高保真度的三维内容，并应用渐进式三维合成策略，显著提高了生成效率和质量，包括解决了 Janus 问题，并在定量和定性评估中证明了优越性。

EfficientDreamer: 高保真和鲁棒的三维创作通过正交视图扩散先验

EfficientDreamer: High-Fidelity and Robust 3D Creation via  Orthogonal-view Diffusion Prior

The view inconsistency problem in score-distilling text-to-3D generation,
also known as the Janus problem, arises from the intrinsic bias of 2D diffusion
models, which leads to the unrealistic generation of 3D objects. In this work,
we explore score-distilling text-to-3D generation and identify the main causes
of the Janus problem. Based on these findings, we propose two approaches to
debias the score-distillation frameworks for robust text-to-3D generation. Our
first approach, called score debiasing, involves gradually increasing the
truncation value for the score estimated by 2D diffusion models throughout the
optimization process. Our second approach, called prompt debiasing, identifies
conflicting words between user prompts and view prompts utilizing a language
model and adjusts the discrepancy between view prompts and object-space camera
poses. Our experimental results show that our methods improve realism by
significantly reducing artifacts and achieve a good trade-off between
faithfulness to the 2D diffusion models and 3D consistency with little
overhead.

研究了文本到 3D 生成中的 Janus 问题，提出了得分去偏置和提示去偏置两种方法来解决该问题，并通过实验证明了这些方法的有效性。