In recent years, 3D models have been utilized in many applications, such as
auto-driver, 3D reconstruction, VR, and AR. However, the scarcity of 3D model
data does not meet its practical demands. Thus, generating high-quality 3D
models efficiently from textual descriptions is a promising but challenging way
to solve this problem. In this paper, inspired by the ability of human beings
to complement visual information details from ambiguous descriptions based on
their own experience, we propose a novel text-3D generation model (T2TD), which
introduces the related shapes or textual information as the prior knowledge to
improve the performance of the 3D generation model. In this process, we first
introduce the text-3D knowledge graph to save the relationship between 3D
models and textual semantic information, which can provide the related shapes
to guide the target 3D model generation. Second, we integrate an effective
causal inference model to select useful feature information from these related
shapes, which removes the unrelated shape information and only maintains
feature information that is strongly relevant to the textual description.
Meanwhile, to effectively integrate multi-modal prior knowledge into textual
information, we adopt a novel multi-layer transformer structure to
progressively fuse related shape and textual information, which can effectively
compensate for the lack of structural information in the text and enhance the
final performance of the 3D generation model. The final experimental results
demonstrate that our approach significantly improves 3D model generation
quality and outperforms the SOTA methods on the text2shape datasets.

本文提出一种新颖的文本生成 3D 模型方法（T2TD），通过引入相关形状或文本信息作为先验知识来提高 3D 模型生成模型的性能，并采用多层变压器结构逐步融合相关形状和文本信息，证明了该方法在 3D 模型生成质量上显着提高，且表现优于现有文本转形状数据集上的 SOTA 方法。

基于先验知识引导的文本 - 三维生成模型

T2TD: Text-3D Generation Model based on Prior Knowledge Guidance

Pretrained models have produced great success in both Computer Vision (CV)
and Natural Language Processing (NLP). This progress leads to learning joint
representations of vision and language pretraining by feeding visual and
linguistic contents into a multi-layer transformer, Visual-Language Pretrained
Models (VLPMs). In this paper, we present an overview of the major advances
achieved in VLPMs for producing joint representations of vision and language.
As the preliminaries, we briefly describe the general task definition and
genetic architecture of VLPMs. We first discuss the language and vision data
encoding methods and then present the mainstream VLPM structure as the core
content. We further summarise several essential pretraining and fine-tuning
strategies. Finally, we highlight three future directions for both CV and NLP
researchers to provide insightful guidance.

本文主要介绍了预训练模型在计算机视觉和自然语言处理中所取得的巨大成功，着重介绍了视觉语言预训练模型 (VLPM) 的重要进展及其结构、预训练和微调策略，并提出了未来三个方向的研究建议。