Federated learning refers to a distributed machine learning paradigm in which
data samples are decentralized and distributed among multiple clients. These
samples may exhibit statistical heterogeneity, which refers to data
distributions are not independent and identical across clients. Additionally,
system heterogeneity, or variations in the computational power of the clients,
introduces biases into federated learning. The combined effects of statistical
and system heterogeneity can significantly reduce the efficiency of federated
optimization. However, the impact of hybrid heterogeneity is not rigorously
discussed. This paper explores how hybrid heterogeneity affects federated
optimization by investigating server-side optimization. The theoretical results
indicate that adaptively maximizing gradient diversity in server update
direction can help mitigate the potential negative consequences of hybrid
heterogeneity. To this end, we introduce a novel server-side gradient-based
optimizer \textsc{FedAWARE} with theoretical guarantees provided. Intensive
experiments in heterogeneous federated settings demonstrate that our proposed
optimizer can significantly enhance the performance of federated learning
across varying degrees of hybrid heterogeneity.

研究了混合异构性如何影响联邦优化，通过调查服务器端优化表明，自适应地最大化梯度多样性在服务器更新方向上可以帮助缓解混合异构性的潜在负面影响，引入了具有理论保证的新型基于梯度的优化器 FedAWARE，通过在异构联邦设置中进行大量实验表明，我们的提议优化器可以显著提升不同程度的混合异构性下联邦学习的性能。