Federated Learning (FL) is a distributed learning scheme to train a shared model across clients. One common and fundamental challenge in FL is that the sets of data across clients could be non-identically distributed and have different sizes. Personalized Federated Learning (PFL) attempts to solve this challenge via locally adapted models. In this work, we present a novel framework for PFL based on hierarchical Bayesian modeling and variational inference. A global model is introduced as a latent variable to augment the joint distribution of clients' parameters and capture the common trends of different clients, optimization is derived based on the principle of maximizing the marginal likelihood and conducted using variational expectation maximization. Our algorithm gives rise to a closed-form estimation of a confidence value which comprises the uncertainty of clients' parameters and local model deviations from the global model. The confidence value is used to weigh clients' parameters in the aggregation stage and adjust the regularization effect of the global model. We evaluate our method through extensive empirical studies on multiple datasets. Experimental results show that our approach obtains competitive results under mild heterogeneous circumstances while significantly outperforming state-of-the-art PFL frameworks in highly heterogeneous settings. Our code is available at https://github.com/JunyiZhu-AI/confidence_aware_PFL.

本文提出的基于分层贝叶斯模型和变分推断的个人化联邦学习算法，引入全局模型作为潜在变量，以捕捉不同客户端之间的公共趋势，使用变分期望最大化技术训练模型并得出包含客户端参数不确定性和本地模型偏差的置信度评估，进而对客户端参数进行加权，优化全局模型的正则化效果，在多个数据集上进行了大量实证研究并证明了其优越性。

可信个性化联邦学习：基于变分期望最大化的方法