3D shape generation aims to produce innovative 3D content adhering to
specific conditions and constraints. Existing methods often decompose 3D shapes
into a sequence of localized components, treating each element in isolation
without considering spatial consistency. As a result, these approaches exhibit
limited versatility in 3D data representation and shape generation, hindering
their ability to generate highly diverse 3D shapes that comply with the
specified constraints. In this paper, we introduce a novel spatial-aware 3D
shape generation framework that leverages 2D plane representations for enhanced
3D shape modeling. To ensure spatial coherence and reduce memory usage, we
incorporate a hybrid shape representation technique that directly learns a
continuous signed distance field representation of the 3D shape using
orthogonal 2D planes. Additionally, we meticulously enforce spatial
correspondences across distinct planes using a transformer-based autoencoder
structure, promoting the preservation of spatial relationships in the generated
3D shapes. This yields an algorithm that consistently outperforms
state-of-the-art 3D shape generation methods on various tasks, including
unconditional shape generation, multi-modal shape completion, single-view
reconstruction, and text-to-shape synthesis.

介绍了一种新型的空间感知的 3D 形状生成框架，利用 2D 平面表示增强 3D 形状建模，通过使用正交 2D 平面直接学习三维形状的连续有符号距离场表示，以确保空间一致性和减少内存使用，并通过基于 Transformer 的自编码结构精确实施不同平面之间的空间对应关系，促进生成的 3D 形状中空间关系的保留，从而在无条件形状生成、多模态形状补全、单视图重建和文本到形状合成等各种任务上始终优于最先进的 3D 形状生成方法。

NeuSDFusion：一种面向空间感知的生成模型用于 3D 形状补全、重构和生成

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion,  Reconstruction, and Generation

This paper proposes an automatic spatially-aware concept discovery approach
using weakly labeled image-text data from shopping websites. We first fine-tune
GoogleNet by jointly modeling clothing images and their corresponding
descriptions in a visual-semantic embedding space. Then, for each attribute
(word), we generate its spatially-aware representation by combining its
semantic word vector representation with its spatial representation derived
from the convolutional maps of the fine-tuned network. The resulting
spatially-aware representations are further used to cluster attributes into
multiple groups to form spatially-aware concepts (e.g., the neckline concept
might consist of attributes like v-neck, round-neck, etc). Finally, we
decompose the visual-semantic embedding space into multiple concept-specific
subspaces, which facilitates structured browsing and attribute-feedback product
retrieval by exploiting multimodal linguistic regularities. We conducted
extensive experiments on our newly collected Fashion200K dataset, and results
on clustering quality evaluation and attribute-feedback product retrieval task
demonstrate the effectiveness of our automatically discovered spatially-aware
concepts.

本文提出了一种利用来自购物网站的弱标签化的图像文本数据进行自动空间感知概念发现的方法，通过将服装图像及其对应描述联合建模，并通过将词汇语义向量表示与卷积地图得出的空间表示相结合生成空间感知表示。我们进一步将空间感知表示用于属性聚类，以形成空间感知概念，最终将视觉语义嵌入空间分解为多种概念特定子空间，以利用多模态语言规律实现结构化浏览和属性反馈产品检索。我们在新收集的 Fashion200K 数据集上进行了大量实验，聚类质量评估和属性反馈产品检索任务的结果证明了我们自动发现的空间感知概念的有效性。