Federated Learning (FL) has surged in prominence due to its capability of
collaborative model training without direct data sharing. However, the vast
disparity in local data distributions among clients, often termed the
non-Independent Identically Distributed (non-IID) challenge, poses a
significant hurdle to FL's generalization efficacy. The scenario becomes even
more complex when not all clients participate in the training process, a common
occurrence due to unstable network connections or limited computational
capacities. This can greatly complicate the assessment of the trained models'
generalization abilities. While a plethora of recent studies has centered on
the generalization gap pertaining to unseen data from participating clients
with diverse distributions, the divergence between the training distributions
of participating clients and the testing distributions of non-participating
ones has been largely overlooked. In response, our paper unveils an
information-theoretic generalization framework for FL. Specifically, it
quantifies generalization errors by evaluating the information entropy of local
distributions and discerning discrepancies across these distributions. Inspired
by our deduced generalization bounds, we introduce a weighted aggregation
approach and a duo of client selection strategies. These innovations aim to
bolster FL's generalization prowess by encompassing a more varied set of client
data distributions. Our extensive empirical evaluations reaffirm the potency of
our proposed methods, aligning seamlessly with our theoretical construct.

我们的论文提出了一种信息论的泛化框架来解决联邦学习中的分布不一致问题，并通过引入加权聚合方法和客户端选择策略来增强泛化能力。