As a privacy-preserving collaborative machine learning paradigm, federated
learning (FL) has attracted significant interest from academia and the industry
alike. To allow each data owner (a.k.a., FL clients) to train a heterogeneous
and personalized local model based on its local data distribution, system
resources and requirements on model structure, the field of model-heterogeneous
personalized federated learning (MHPFL) has emerged. Existing MHPFL approaches
either rely on the availability of a public dataset with special
characteristics to facilitate knowledge transfer, incur high computation and
communication costs, or face potential model leakage risks. To address these
limitations, we propose a model-heterogeneous personalized Federated learning
approach based on feature Extractor Sharing (pFedES). It incorporates a small
homogeneous feature extractor into each client's heterogeneous local model.
Clients train them via the proposed iterative learning method to enable the
exchange of global generalized knowledge and local personalized knowledge. The
small local homogeneous extractors produced after local training are uploaded
to the FL server and for aggregation to facilitate easy knowledge sharing among
clients. We theoretically prove that pFedES can converge over wall-to-wall
time. Extensive experiments on two real-world datasets against six
state-of-the-art methods demonstrate that pFedES builds the most accurate
model, while incurring low communication and computation costs. Compared with
the best-performing baseline, it achieves 1.61% higher test accuracy, while
reducing communication and computation costs by 99.6% and 82.9%, respectively.

提出了基于特征提取器共享的异构个性化联邦学习方法 (pFedES)，该方法在不同客户端的本地模型中引入小型同质特征提取器，通过迭代学习方法进行训练，实现全局泛化知识和本地个性化知识的交换和共享，其理论上证明了 pFedES 能够在墙时收敛。在两个真实数据集上的广泛实验中，与六种最先进的方法相比，pFedES 构建了最准确的模型，同时通信和计算成本较低，与最佳基准相比，测试准确性提高了 1.61%，通信和计算成本分别降低了 99.6% 和 82.9%。