Federated Learning (FL) aims to train machine learning models for multiple
clients without sharing their own private data. Due to the heterogeneity of
clients' local data distribution, recent studies explore the personalized FL
that learns and deploys distinct local models with the help of auxiliary global
models. However, the clients can be heterogeneous in terms of not only local
data distribution, but also their computation and communication resources. The
capacity and efficiency of personalized models are restricted by the
lowest-resource clients, leading to sub-optimal performance and limited
practicality of personalized FL. To overcome these challenges, we propose a
novel approach named pFedGate for efficient personalized FL by adaptively and
efficiently learning sparse local models. With a lightweight trainable gating
layer, pFedGate enables clients to reach their full potential in model capacity
by generating different sparse models accounting for both the heterogeneous
data distributions and resource constraints. Meanwhile, the computation and
communication efficiency are both improved thanks to the adaptability between
the model sparsity and clients' resources. Further, we theoretically show that
the proposed pFedGate has superior complexity with guaranteed convergence and
generalization error. Extensive experiments show that pFedGate achieves
superior global accuracy, individual accuracy and efficiency simultaneously
over state-of-the-art methods. We also demonstrate that pFedGate performs
better than competitors in the novel clients participation and partial clients
participation scenarios, and can learn meaningful sparse local models adapted
to different data distributions.

提出一种名为 pFedGate 的方法，通过自适应和高效地学习稀疏本地模型来实现高效个性化联邦学习，并通过轻量级可训练的门控层使客户端生成不同的稀疏模型，同时考虑到异构的数据分布和资源限制，实现了计算和通信的效率提高，理论上的收敛性和泛化误差更优越，实验证明该方法相对于同类方法能够同时实现更高的全局准确性、个体准确性和效率，并能够适应不同的数据分布。