Federated learning enables joint training of machine learning models from
distributed clients without sharing their local data. One key challenge in
federated learning is to handle non-identically distributed data across the
clients, which leads to deteriorated model training performances. Prior works
in this line of research mainly focus on utilizing last-step global model
parameters/gradients or the linear combinations of the past model
parameters/gradients, which do not fully exploit the potential of global
information from the model training trajectory. In this paper, we propose a
novel federated learning framework with projected trajectory regularization
(FedPTR) for tackling the data heterogeneity issue, which proposes a unique way
to better extract the essential global information from the model training
trajectory. Specifically, FedPTR allows local clients or the server to optimize
an auxiliary (synthetic) dataset that mimics the learning dynamics of the
recent model update and utilizes it to project the next-step model trajectory
for local training regularization. We conduct rigorous theoretical analysis for
our proposed framework under nonconvex stochastic settings to verify its fast
convergence under heterogeneous data distributions. Experiments on various
benchmark datasets and non-i.i.d. settings validate the effectiveness of our
proposed framework.

提出了一种新颖的联邦学习框架（FedPTR），通过采用轨迹正则化来解决分布式学习中数据异质性问题，利用模型训练轨迹中的全局信息进行本地训练规范化，经过理论分析和实验证明其快速收敛并在异质数据分布下具有良好效果。