Zero-shot 3D point cloud understanding can be achieved via 2D Vision-Language
Models (VLMs). Existing strategies directly map Vision-Language Models from 2D
pixels of rendered or captured views to 3D points, overlooking the inherent and
expressible point cloud geometric structure. Geometrically similar or close
regions can be exploited for bolstering point cloud understanding as they are
likely to share semantic information. To this end, we introduce the first
training-free aggregation technique that leverages the point cloud's 3D
geometric structure to improve the quality of the transferred Vision-Language
Models. Our approach operates iteratively, performing local-to-global
aggregation based on geometric and semantic point-level reasoning. We benchmark
our approach on three downstream tasks, including classification, part
segmentation, and semantic segmentation, with a variety of datasets
representing both synthetic/real-world, and indoor/outdoor scenarios. Our
approach achieves new state-of-the-art results in all benchmarks. We will
release the source code publicly.

通过利用点云的几何结构改进传输的视觉 - 语言模型，我们提出了首个无需训练的聚合技术，通过几何和语义的点级推理，在分类、部件分割和语义分割等三个任务中取得了新的技术优势。

基于几何驱动的零样本三维点云理解聚合

Geometrically-driven Aggregation for Zero-shot 3D Point Cloud  Understanding

We address the problem of learning on sets of features, motivated by the need
of performing pooling operations in long biological sequences of varying sizes,
with long-range dependencies, and possibly few labeled data. To address this
challenging task, we introduce a parametrized representation of fixed size,
which embeds and then aggregates elements from a given input set according to
the optimal transport plan between the set and a trainable reference. Our
approach scales to large datasets and allows end-to-end training of the
reference, while also providing a simple unsupervised learning mechanism with
small computational cost. Our aggregation technique admits two useful
interpretations: it may be seen as a mechanism related to attention layers in
neural networks, or it may be seen as a scalable surrogate of a classical
optimal transport-based kernel. We experimentally demonstrate the effectiveness
of our approach on biological sequences, achieving state-of-the-art results for
protein fold recognition and detection of chromatin profiles tasks, and, as a
proof of concept, we show promising results for processing natural language
sequences. We provide an open-source implementation of our embedding that can
be used alone or as a module in larger learning models at
this https URL

本文提出了一种基于优化传输计划思想的特征集聚合技术，可用于长生物序列的池化操作，具有快速、简单、可扩展等优点，并在蛋白质折叠识别、染色质图谱检测等生物学任务上取得了最先进的效果。