The popularity of pre-trained large models has revolutionized downstream
tasks across diverse fields, such as language, vision, and multi-modality. To
minimize the adaption cost for downstream tasks, many Parameter-Efficient
Fine-Tuning (PEFT) techniques are proposed for language and 2D image
pre-trained models. However, the specialized PEFT method for 3D pre-trained
models is still under-explored. To this end, we introduce Point-PEFT, a novel
framework for adapting point cloud pre-trained models with minimal learnable
parameters. Specifically, for a pre-trained 3D model, we freeze most of its
parameters, and only tune the newly added PEFT modules on downstream tasks,
which consist of a Point-prior Prompt and a Geometry-aware Adapter. The
Point-prior Prompt adopts a set of learnable prompt tokens, for which we
propose to construct a memory bank with domain-specific knowledge, and utilize
a parameter-free attention to enhance the prompt tokens. The Geometry-aware
Adapter aims to aggregate point cloud features within spatial neighborhoods to
capture fine-grained geometric information through local interactions.
Extensive experiments indicate that our Point-PEFT can achieve better
performance than the full fine-tuning on various downstream tasks, while using
only 5% of the trainable parameters, demonstrating the efficiency and
effectiveness of our approach. Code will be released at
this https URL

通过 Point-PEFT 框架，我们能够在只使用可训练参数的 5% 的情况下，实现优于完全微调的性能，充分展示了我们方法的高效性和有效性。