Large-scale pre-trained vision models (PVMs) have shown great potential for
adaptability across various downstream vision tasks. However, with
state-of-the-art PVMs growing to billions or even trillions of parameters, the
standard full fine-tuning paradigm is becoming unsustainable due to high
computational and storage demands. In response, researchers are exploring
parameter-efficient fine-tuning (PEFT), which seeks to exceed the performance
of full fine-tuning with minimal parameter modifications. This survey provides
a comprehensive overview and future directions for visual PEFT, offering a
systematic review of the latest advancements. First, we provide a formal
definition of PEFT and discuss model pre-training methods. We then categorize
existing methods into three categories: addition-based, partial-based, and
unified-based. Finally, we introduce the commonly used datasets and
applications and suggest potential future research challenges. A comprehensive
collection of resources is available at
this https URL

规模化预训练视觉模型（PVMs）在各种下游视觉任务中表现出很大的适应性。然而，随着最先进的 PVMs 达到数十亿甚至数万亿参数，传统的完全微调范式变得难以持续，因为其需要巨大的计算和存储需求。为了应对这一挑战，研究人员正在探索参数高效的微调（PEFT），旨在通过最小的参数修改超越完全微调的性能。本调查提供了对视觉 PEFT 的综合概述和未来方向，对最新的进展进行了系统回顾。首先，我们提供了 PEFT 的正式定义，并讨论了模型预训练方法。然后，我们将现有方法分为三类：基于添加的、基于部分的和基于统一的。最后，我们介绍了常用的数据集和应用，并提出了未来研究的潜在挑战。所有相关资源可以在该链接中找到。

预训练视觉模型的参数高效微调：一项调查

Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey

Large-scale pre-trained vision foundation models, such as CLIP, have become
de facto backbones for various vision tasks. However, due to their black-box
nature, understanding the underlying rules behind these models' predictions and
controlling model behaviors have remained open challenges. We present a
framework for interpreting vision transformer's latent tokens with natural
language. Given a latent token, our framework retains its semantic information
to the final layer using transformer's local operations and retrieves the
closest text for explanation. Our approach enables understanding of model
visual reasoning procedure without needing additional model training or data
collection. Based on the obtained interpretations, our framework allows for
model editing that controls model reasoning behaviors and improves model
robustness against biases and spurious correlations.

基于大规模预训练视觉模型，如 CLIP，的本文的研究主要集中在模型的解释和控制，通过理解模型的潜在令牌和进行模型编辑来改进模型的推理行为和鲁棒性。