Federated learning (FL) facilitates a privacy-preserving neural network
training paradigm through collaboration between edge clients and a central
server. One significant challenge is that the distributed data is not
independently and identically distributed (non-IID), typically including both
intra-domain and inter-domain heterogeneity. However, recent research is
limited to simply using averaged signals as a form of regularization and only
focusing on one aspect of these non-IID challenges. Given these limitations,
this paper clarifies these two non-IID challenges and attempts to introduce
cluster representation to address them from both local and global perspectives.
Specifically, we propose a dual-clustered feature contrast-based FL framework
with dual focuses. First, we employ clustering on the local representations of
each client, aiming to capture intra-class information based on these local
clusters at a high level of granularity. Then, we facilitate cross-client
knowledge sharing by pulling the local representation closer to clusters shared
by clients with similar semantics while pushing them away from clusters with
dissimilar semantics. Second, since the sizes of local clusters belonging to
the same class may differ for each client, we further utilize clustering on the
global side and conduct averaging to create a consistent global signal for
guiding each local training in a contrastive manner. Experimental results on
multiple datasets demonstrate that our proposal achieves comparable or superior
performance gain under intra-domain and inter-domain heterogeneity.

通过引入聚类表示、局部聚类和全局信号等方法，本研究提出了一个双聚类特征对比的联邦学习框架，旨在解决非独立同分布数据的挑战，以跨客户知识共享和一致的全局指导为重点，实验结果表明，在域内和域间异质性下，该框架取得了可比或更高的性能提升。

FedCCL：基于领域异质性的联邦双聚类特征对比

FedCCL: Federated Dual-Clustered Feature Contrast Under Domain  Heterogeneity

Existing research has either adapted the Probably Approximately Correct (PAC)
Bayesian framework for federated learning (FL) or used information-theoretic
PAC-Bayesian bounds while introducing their theorems, but few considering the
non-IID challenges in FL. Our work presents the first non-vacuous federated
PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique
prior knowledge for each client and variable aggregation weights. We also
introduce an objective function and an innovative Gibbs-based algorithm for the
optimization of the derived bound. The results are validated on real-world
datasets.

我们的研究针对分布不均匀的本地数据，提出了第一个非平凡的针对非独立同分布挑战的联邦 PAC-Bayesian 界限，并使用独特的先验知识和可变聚合权重。我们还引入了一个目标函数和一种创新的基于 Gibbs 的算法来优化该界限，并在真实数据集上验证了结果。