In contrast to training traditional machine learning (ML) models in data
centers, federated learning (FL) trains ML models over local datasets contained
on resource-constrained heterogeneous edge devices. Existing FL algorithms aim
to learn a single global model for all participating devices, which may not be
helpful to all devices participating in the training due to the heterogeneity
of the data across the devices. Recently, Hanzely and Richtárik (2020)
proposed a new formulation for training personalized FL models aimed at
balancing the trade-off between the traditional global model and the local
models that could be trained by individual devices using their private data
only. They derived a new algorithm, called Loopless Gradient Descent (L2GD), to
solve it and showed that this algorithms leads to improved communication
complexity guarantees in regimes when more personalization is required. In this
paper, we equip their L2GD algorithm with a bidirectional compression mechanism
to further reduce the communication bottleneck between the local devices and
the server. Unlike other compression-based algorithms used in the FL-setting,
our compressed L2GD algorithm operates on a probabilistic communication
protocol, where communication does not happen on a fixed schedule. Moreover,
our compressed L2GD algorithm maintains a similar convergence rate as vanilla
SGD without compression. To empirically validate the efficiency of our
algorithm, we perform diverse numerical experiments on both convex and
non-convex problems and using various compression techniques.

本文提出通过在个体边缘设备上训练个性化模型以取代传统机器学习模型，以解决数据异质性带来的问题，并对新算法 Loopless Gradient Descent 进行了压缩优化，实验结果表明该算法比其他基于压缩的算法运行效率更高且收敛速度不低于 vanilla SGD。