In recent years, personalized federated learning (pFL) has attracted
increasing attention for its potential in dealing with statistical
heterogeneity among clients. However, the state-of-the-art pFL methods rely on
model parameters aggregation at the server side, which require all models to
have the same structure and size, and thus limits the application for more
heterogeneous scenarios. To deal with such model constraints, we exploit the
potentials of heterogeneous model settings and propose a novel training
framework to employ personalized models for different clients. Specifically, we
formulate the aggregation procedure in original pFL into a personalized group
knowledge transfer training algorithm, namely, KT-pFL, which enables each
client to maintain a personalized soft prediction at the server side to guide
the others' local training. KT-pFL updates the personalized soft prediction of
each client by a linear combination of all local soft predictions using a
knowledge coefficient matrix, which can adaptively reinforce the collaboration
among clients who own similar data distribution. Furthermore, to quantify the
contributions of each client to others' personalized training, the knowledge
coefficient matrix is parameterized so that it can be trained simultaneously
with the models. The knowledge coefficient matrix and the model parameters are
alternatively updated in each round following the gradient descent way.
Extensive experiments on various datasets (EMNIST, Fashion\_MNIST, CIFAR-10)
are conducted under different settings (heterogeneous models and data
distributions). It is demonstrated that the proposed framework is the first
federated learning paradigm that realizes personalized model training via
parameterized group knowledge transfer while achieving significant performance
gain comparing with state-of-the-art algorithms.

提出了一个基于个性化模型的联邦学习框架来应对异构模型的限制，并使用参数化群体知识转移训练算法来使客户端间自适应强化协作，从而实现个性化模型训练，并与现有方法进行比较得到显著的性能提升。