We introduce a new algorithm - Artemis - tackling the problem of learning in a distributed framework with communication constraints. Several workers perform the optimization process using a central server to aggregate their computation. To alleviate the communication cost, Artemis compresses the information sent in both directions (from the workers to the server and conversely) combined with a memory mechanism. It improves on existing quantized federated learning algorithms that only consider unidirectional compression (to the server), or use very strong assumptions on the compression operator. We provide fast rates of convergence (linear up to a threshold) under weak assumptions on the stochastic gradients (noise's variance bounded only at optimal point) in non-i.i.d. setting, highlight the impact of memory for unidirectional and bidirectional compression, analyze Polyak-Ruppert averaging, use convergence in distribution to obtain a lower bound of the asymptotic variance that highlights practical limits of compression, and provide experimental results to demonstrate the validity of our analysis.

本研究引入了一种名为阿尔忒弥斯的框架，旨在解决分布式或联邦学习中的通信限制和设备部分参与的问题。该框架可在双向压缩信息方面改善现有算法，并提供了快速收敛率和针对部分参与设备挑战的解决方案。

异构环境中双向压缩用于部分参与的分布式或联邦学习: 紧致的收敛保证