This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Importance-Weighted Empirical Risk Minimization (FTW-ERM) along with improving density ratio matching methods without requiring perfect knowledge of the supremum over true ratios. We also propose the communication-efficient variant FITW-ERM with the same level of privacy guarantees as those of classical ERM in FL. We theoretically show that FTW-ERM achieves smaller generalization error than classical ERM under certain settings. Experimental results demonstrate the superiority of FTW-ERM over existing FL baselines in challenging imbalanced federated settings in terms of data distribution shifts across clients.

本文提出了一种新的全局模型训练范式FTW-ERM，利用提高密度比匹配方法来处理联邦学习中的客户端内部和客户端间的协变量移位，并提出了与经典ERM在FL中相同隐私保证水平的低通信效率的FITW-ERM变体。理论上表明，在某些情况下，FTW-ERM比经典ERM具有更小的泛化误差。实验结果表明，在具有挑战性的不平衡联邦设置中，FTW-ERM优于现有联邦学习基线，因为数据分布在客户端之间存在移位。

具有泛化保证的协变量移位下的联邦学习