In recent years, zero-shot learning has attracted the focus of many
researchers, due to its flexibility and generality. Many approaches have been
proposed to achieve the zero-shot classification of the point clouds for 3D
object understanding, following the schema of CLIP. However, in the real world,
the point clouds could be extremely sparse, dramatically limiting the
effectiveness of the 3D point cloud encoders, and resulting in the misalignment
of point cloud features and text embeddings. To the point cloud encoders to fit
the extremely sparse point clouds without re-running the pre-training procedure
which could be time-consuming and expensive, in this work, we propose an
unsupervised model adaptation approach to enhance the point cloud encoder for
the extremely sparse point clouds. We propose a novel fused-cross attention
layer that expands the pre-trained self-attention layer with additional
learnable tokens and attention blocks, which effectively modifies the point
cloud features while maintaining the alignment between point cloud features and
text embeddings. We also propose a complementary learning-based
self-distillation schema that encourages the modified features to be pulled
apart from the irrelevant text embeddings without overfitting the feature space
to the observed text embeddings. Extensive experiments demonstrate that the
proposed approach effectively increases the zero-shot capability on extremely
sparse point clouds, and overwhelms other state-of-the-art model adaptation
approaches.

在本论文中，我们提出了一种无监督的模型自适应方法来增强用于处理极其稀疏点云的点云编码器，通过引入融合交叉注意力层和互补学习的自蒸馏方案，有效地改进点云特征并保持与文本嵌入的对齐，以提高处理这种极其稀疏点云的零样本分类能力。

ESP-Zero: 无监督增强极稀疏点云的零样本分类

ESP-Zero: Unsupervised enhancement of zero-shot classification for  Extremely Sparse Point cloud

Unsupervised domain adaptation aims to align a labeled source domain and an
unlabeled target domain, but it requires to access the source data which often
raises concerns in data privacy, data portability and data transmission
efficiency. We study unsupervised model adaptation (UMA), or called
Unsupervised Domain Adaptation without Source Data, an alternative setting that
aims to adapt source-trained models towards target distributions without
accessing source data. To this end, we design an innovative historical
contrastive learning (HCL) technique that exploits historical source hypothesis
to make up for the absence of source data in UMA. HCL addresses the UMA
challenge from two perspectives. First, it introduces historical contrastive
instance discrimination (HCID) that learns from target samples by contrasting
their embeddings which are generated by the currently adapted model and the
historical models. With the historical models, HCID encourages UMA to learn
instance-discriminative target representations while preserving the source
hypothesis. Second, it introduces historical contrastive category
discrimination (HCCD) that pseudo-labels target samples to learn
category-discriminative target representations. Specifically, HCCD re-weights
pseudo labels according to their prediction consistency across the current and
historical models. Extensive experiments show that HCL outperforms and
state-of-the-art methods consistently across a variety of visual tasks and
setups.

本研究提出了一种无监督模型自适应的历史对比学习技术，通过历史模型的假设来弥补源数据中的缺失，从而实现将源训练模型适应到目标分布的无监督域适应。实验结果表明，该方法在各种视觉任务和设置下一致优于现有技术。