We present a unified framework for analyzing local SGD methods in the convex and strongly convex regimes for distributed/federated training of supervised machine learning models. We recover several known methods as a special case of our general framework, including Local-SGD/FedAvg, SCAFFOLD, and several variants of SGD not originally designed for federated learning. Our framework covers both the identical and heterogeneous data settings, supports both random and deterministic number of local steps, and can work with a wide array of local stochastic gradient estimators, including shifted estimators which are able to adjust the fixed points of local iterations for faster convergence. As an application of our framework, we develop multiple novel FL optimizers which are superior to existing methods. In particular, we develop the first linearly converging local SGD method which does not require any data homogeneity or other strong assumptions.

该论文提出了一种统一框架，用于在凸性和强凸性条件下分析本地SGD方法，适用于监督机器学习模型的分布式/联邦训练。作为该框架的应用，作者开发了多个新型优化器，特别是开发了第一个线性收敛的本地SGD方法，不需要任何数据同质性或其他强假设。

局部 SGD：统一理论和新高效方法