In federated optimization, heterogeneity in the clients' local datasets and
computation speeds results in large variations in the number of local updates
performed by each client in each communication round. Naive weighted
aggregation of such models causes objective inconsistency, that is, the global
model converges to a stationary point of a mismatched objective function which
can be arbitrarily different from the true objective. This paper provides a
general framework to analyze the convergence of federated heterogeneous
optimization algorithms. It subsumes previously proposed methods such as FedAvg
and FedProx and provides the first principled understanding of the solution
bias and the convergence slowdown due to objective inconsistency. Using
insights from this analysis, we propose FedNova, a normalized averaging method
that eliminates objective inconsistency while preserving fast error
convergence.

本文为联邦异构优化提供了一个通用的框架，分析了算法的收敛性，解决了不同本地数据集和计算速度引起的目标不一致性问题，并提出了 FedNova 方法，实现了快速错误收敛并消除了目标不一致性。