This paper focuses on reducing the communication cost of federated learning
by exploring generalization bounds and representation learning. We first
characterize a tighter generalization bound for one-round federated learning
based on local clients' generalizations and heterogeneity of data distribution
(non-iid scenario). We also characterize a generalization bound in R-round
federated learning and its relation to the number of local updates (local
stochastic gradient descents (SGDs)). Then, based on our generalization bound
analysis and our representation learning interpretation of this analysis, we
show for the first time that less frequent aggregations, hence more local
updates, for the representation extractor (usually corresponds to initial
layers) leads to the creation of more generalizable models, particularly for
non-iid scenarios. We design a novel Federated Learning with Adaptive Local
Steps (FedALS) algorithm based on our generalization bound and representation
learning analysis. FedALS employs varying aggregation frequencies for different
parts of the model, so reduces the communication cost. The paper is followed
with experimental results showing the effectiveness of FedALS.

本文通过探索泛化界限和表示学习来减少联邦学习的通信成本，首次展示了在非独立同分布的情况下，更少的聚合次数，更多的本地更新可产生更具一般化能力的模型，设计了基于泛化界限和表示学习分析的自适应本地步骤联邦学习算法 FedALS，通过对模型不同部分采用不同的聚合频率来降低通信成本，并通过实验结果证明了 FedALS 的有效性。