Federated Learning (FL) aggregates locally trained models from individual
clients to construct a global model. While FL enables learning a model with
data privacy, it often suffers from significant performance degradation when
client data distributions are heterogeneous. Many previous FL algorithms have
addressed this issue by introducing various proximal restrictions. These
restrictions aim to encourage global alignment by constraining the deviation of
local learning from the global objective. However, they inherently limit local
learning by interfering with the original local objectives. Recently, an
alternative approach has emerged to improve local learning generality. By
obtaining local models within a smooth loss landscape, this approach mitigates
conflicts among different local objectives of the clients. Yet, it does not
ensure stable global alignment, as local learning does not take the global
objective into account. In this study, we propose Federated Stability on
Learning (FedSoL), which combines both the concepts of global alignment and
local generality. In FedSoL, the local learning seeks a parameter region robust
against proximal perturbations. This strategy introduces an implicit proximal
restriction effect in local learning while maintaining the original local
objective for parameter update. Our experiments show that FedSoL consistently
achieves state-of-the-art performance on various setups.

通过结合全局对齐和局部普遍性的概念，我们提出了名为 FedSoL 的联邦稳定学习方法，该方法在寻求抵御邻近扰动的参数区域时，既可维持原始局部目标进行参数更新，又能在局部学习中引入一种隐式的邻近限制效果。进行了一系列实验表明，FedSoL 在各种场景下始终保持最先进的性能。

FedSoL: 在联合学习中桥接全球对齐和本地通用性

FedSoL: Bridging Global Alignment and Local Generality in Federated  Learning

Federated Learning (FL) is a promising framework for performing
privacy-preserving, distributed learning with a set of clients. However, the
data distribution among clients often exhibits non-IID, i.e., distribution
shift, which makes efficient optimization difficult. To tackle this problem,
many FL algorithms focus on mitigating the effects of data heterogeneity across
clients by increasing the performance of the global model. However, almost all
algorithms leverage Empirical Risk Minimization (ERM) to be the local
optimizer, which is easy to make the global model fall into a sharp valley and
increase a large deviation of parts of local clients. Therefore, in this paper,
we revisit the solutions to the distribution shift problem in FL with a focus
on local learning generality. To this end, we propose a general, effective
algorithm, \texttt{FedSAM}, based on Sharpness Aware Minimization (SAM) local
optimizer, and develop a momentum FL algorithm to bridge local and global
models, \texttt{MoFedSAM}. Theoretically, we show the convergence analysis of
these two algorithms and demonstrate the generalization bound of
\texttt{FedSAM}. Empirically, our proposed algorithms substantially outperform
existing FL studies and significantly decrease the learning deviation.

本文提出了一种名为 FedSAM 的基于 Sharpness Aware Minimization (SAM) 本地优化器的泛化学习算法，以解决 FL 中的数据异构性问题，并在理论上证明了其几何级数的收敛性和泛化边界。在实证方面，我们的算法大大优于现有 FL 研究并显著降低了学习偏差。

基于锐度感知最小化实现的广义联邦学习

Generalized Federated Learning via Sharpness Aware Minimization

Federated learning (FL) is a promising strategy for performing
privacy-preserving, distributed learning with a network of clients (i.e., edge
devices). However, the data distribution among clients is often non-IID in
nature, making efficient optimization difficult. To alleviate this issue, many
FL algorithms focus on mitigating the effects of data heterogeneity across
clients by introducing a variety of proximal terms, some incurring considerable
compute and/or memory overheads, to restrain local updates with respect to the
global model. Instead, we consider rethinking solutions to data heterogeneity
in FL with a focus on local learning generality rather than proximal
restriction. To this end, we first present a systematic study informed by
second-order indicators to better understand algorithm effectiveness in FL.
Interestingly, we find that standard regularization methods are surprisingly
strong performers in mitigating data heterogeneity effects. Based on our
findings, we further propose a simple and effective method, FedAlign, to
overcome data heterogeneity and the pitfalls of previous methods. FedAlign
achieves competitive accuracy with state-of-the-art FL methods across a variety
of settings while minimizing computation and memory overhead. Code is available
at this https URL

本文提出了一种叫做 FedAlign 的基于局部学习广泛性而非接近约束的数据异构性联邦学习解决方案，其可在不引入大量计算和内存开销的情况下实现与最先进的 FL 方法相当的精度。