Adapting Foundation Models (FMs) for downstream tasks through Federated
Learning (FL) emerges a promising strategy for protecting data privacy and
valuable FMs. Existing methods fine-tune FM by allocating sub-FM to clients in
FL, however, leading to suboptimal performance due to insufficient tuning and
inevitable error accumulations of gradients. In this paper, we propose
Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation
in downstream tasks through FL by two key modules. First, the sub-FM
construction module employs a layer-wise compression approach, facilitating
comprehensive FM fine-tuning across all layers by emphasizing those crucial
neurons. Second, the sub-FM alignment module conducts a two-step
distillations-layer-level and neuron-level-before and during FL fine-tuning
respectively, to reduce error of gradient by accurately aligning sub-FM with FM
under theoretical guarantees. Experimental results on seven commonly used
datasets (i.e., four text and three vision) demonstrate the superiority of
FedPFT.

通过联邦学习策略对基础模型进行适应性调整，以保护数据隐私和有价值的基础模型，提出了一种新方法 Federated Proxy Fine-Tuning (FedPFT)，通过两个关键模块增强联邦学习下游任务中的基础模型的适应性。实验证据表明 FedPFT 方法的优越性。