Federated Learning (FL) has emerged as a promising distributed learning
paradigm that enables multiple clients to learn a global model collaboratively
without sharing their private data. However, the effectiveness of FL is highly
dependent on the quality of the data that is being used for training. In
particular, data heterogeneity issues, such as label distribution skew and
feature skew, can significantly impact the performance of FL. Previous studies
in FL have primarily focused on addressing label distribution skew data
heterogeneity, while only a few recent works have made initial progress in
tackling feature skew issues. Notably, these two forms of data heterogeneity
have been studied separately and have not been well explored within a unified
FL framework. To address this gap, we propose Fed-CO$_{2}$, a universal FL
framework that handles both label distribution skew and feature skew within a
\textbf{C}ooperation mechanism between the \textbf{O}nline and \textbf{O}ffline
models. Specifically, the online model learns general knowledge that is shared
among all clients, while the offline model is trained locally to learn the
specialized knowledge of each individual client. To further enhance model
cooperation in the presence of feature shifts, we design an intra-client
knowledge transfer mechanism that reinforces mutual learning between the online
and offline models, and an inter-client knowledge transfer mechanism to
increase the models' domain generalization ability. Extensive experiments show
that our Fed-CO$_{2}$ outperforms a wide range of existing personalized
federated learning algorithms in terms of handling label distribution skew and
feature skew, both individually and collectively. The empirical results are
supported by our convergence analyses in a simplified setting.

我们提出了 Fed-CO2，这是一个统一的联邦学习框架，用于处理标签分布倾斜和特征倾斜，通过在线和离线模型之间的合作机制来增强模型合作，同时设计了内部客户知识传递机制和客户间知识传递机制来增强模型之间的领域泛化能力。在处理标签分布倾斜和特征倾斜方面，我们的 Fed-CO2 在个体和整体上均优于现有的个性化联邦学习算法。