In recent years, zero-shot learning has attracted the focus of many
researchers, due to its flexibility and generality. Many approaches have been
proposed to achieve the zero-shot classification of the point clouds for 3D
object understanding, following the schema of CLIP. However, in the real world,
the point clouds could be extremely sparse, dramatically limiting the
effectiveness of the 3D point cloud encoders, and resulting in the misalignment
of point cloud features and text embeddings. To the point cloud encoders to fit
the extremely sparse point clouds without re-running the pre-training procedure
which could be time-consuming and expensive, in this work, we propose an
unsupervised model adaptation approach to enhance the point cloud encoder for
the extremely sparse point clouds. We propose a novel fused-cross attention
layer that expands the pre-trained self-attention layer with additional
learnable tokens and attention blocks, which effectively modifies the point
cloud features while maintaining the alignment between point cloud features and
text embeddings. We also propose a complementary learning-based
self-distillation schema that encourages the modified features to be pulled
apart from the irrelevant text embeddings without overfitting the feature space
to the observed text embeddings. Extensive experiments demonstrate that the
proposed approach effectively increases the zero-shot capability on extremely
sparse point clouds, and overwhelms other state-of-the-art model adaptation
approaches.

在本论文中，我们提出了一种无监督的模型自适应方法来增强用于处理极其稀疏点云的点云编码器，通过引入融合交叉注意力层和互补学习的自蒸馏方案，有效地改进点云特征并保持与文本嵌入的对齐，以提高处理这种极其稀疏点云的零样本分类能力。