3D shape generation from text is a fundamental task in 3D representation
learning. The text-shape pairs exhibit a hierarchical structure, where a
general text like "chair" covers all 3D shapes of the chair, while more
detailed prompts refer to more specific shapes. Furthermore, both text and 3D
shapes are inherently hierarchical structures. However, existing Text2Shape
methods, such as SDFusion, do not exploit that. In this work, we propose
HyperSDFusion, a dual-branch diffusion model that generates 3D shapes from a
given text. Since hyperbolic space is suitable for handling hierarchical data,
we propose to learn the hierarchical representations of text and 3D shapes in
hyperbolic space. First, we introduce a hyperbolic text-image encoder to learn
the sequential and multi-modal hierarchical features of text in hyperbolic
space. In addition, we design a hyperbolic text-graph convolution module to
learn the hierarchical features of text in hyperbolic space. In order to fully
utilize these text features, we introduce a dual-branch structure to embed text
features in 3D feature space. At last, to endow the generated 3D shapes with a
hierarchical structure, we devise a hyperbolic hierarchical loss. Our method is
the first to explore the hyperbolic hierarchical representation for
text-to-shape generation. Experimental results on the existing text-to-shape
paired dataset, Text2Shape, achieved state-of-the-art results.

提出了一种基于超几何空间的分层文本到形状生成模型，利用超几何空间学习文本和 3D 形状的分层表示，并通过引入双分支结构嵌入文本特征于 3D 特征空间，最终得到具有分层结构的生成 3D 形状，实验结果显示在现有文本到形状数据集上取得了最先进的效果。

超级 SDFusion: 提升 3D 文本到形状生成的语言和几何层次结构的桥接

HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry  for Enhanced 3D Text2Shape Generation

Current state-of-the-art methods for text-to-shape generation either require
supervised training using a labeled dataset of pre-defined 3D shapes, or
perform expensive inference-time optimization of implicit neural
representations. In this work, we present ZeroForge, an approach for zero-shot
text-to-shape generation that avoids both pitfalls. To achieve open-vocabulary
shape generation, we require careful architectural adaptation of existing
feed-forward approaches, as well as a combination of data-free CLIP-loss and
contrastive losses to avoid mode collapse. Using these techniques, we are able
to considerably expand the generative ability of existing feed-forward
text-to-shape models such as CLIP-Forge. We support our method via extensive
qualitative and quantitative evaluations

本文介绍一种名为 ZeroForge 的零样本学习方法，可避免依赖标记的 3D 形状数据集和推断时间昂贵的隐式神经表达优化，通过结合 CLIP-loss 和对比损失等技术，扩大了现有前馈文本到形状模型如 CLIP-Forge 的生成能力，并使用广泛的定性和定量评估支持了这种方法。

ZeroForge: 无需 3D 监督的前向文本到形状

ZeroForge: Feedforward Text-to-Shape Without 3D Supervision

Generating shapes using natural language can enable new ways of imagining and
creating the things around us. While significant recent progress has been made
in text-to-image generation, text-to-shape generation remains a challenging
problem due to the unavailability of paired text and shape data at a large
scale. We present a simple yet effective method for zero-shot text-to-shape
generation that circumvents such data scarcity. Our proposed method, named
CLIP-Forge, is based on a two-stage training process, which only depends on an
unlabelled shape dataset and a pre-trained image-text network such as CLIP. Our
method has the benefits of avoiding expensive inference time optimization, as
well as the ability to generate multiple shapes for a given text. We not only
demonstrate promising zero-shot generalization of the CLIP-Forge model
qualitatively and quantitatively, but also provide extensive comparative
evaluations to better understand its behavior.

本研究提出了一种名为 CLIP-Forge 的简单而有效的方法，该方法是基于文本和形状数据不可用的情况下进行的零样本文本生成图形的二阶段培训过程。该方法不仅具有避免推理时间优化的优势，而且能够为给定的文本生成多个形状。实验结果表明，该模型具有良好的零样本泛化能力，并对其行为进行了广泛的比较评估。