Decentralized learning is crucial in supporting on-device learning over large
distributed datasets, eliminating the need for a central server. However, the
communication overhead remains a major bottleneck for the practical realization
of such decentralized setups. To tackle this issue, several algorithms for
decentralized training with compressed communication have been proposed in the
literature. Most of these algorithms introduce an additional hyper-parameter
referred to as consensus step-size which is tuned based on the compression
ratio at the beginning of the training. In this work, we propose AdaGossip, a
novel technique that adaptively adjusts the consensus step-size based on the
compressed model differences between neighboring agents. We demonstrate the
effectiveness of the proposed method through an exhaustive set of experiments
on various Computer Vision datasets (CIFAR-10, CIFAR-100, Fashion MNIST,
Imagenette, and ImageNet), model architectures, and network topologies. Our
experiments show that the proposed method achieves superior performance
($0-2\%$ improvement in test accuracy) compared to the current state-of-the-art
method for decentralized learning with communication compression.

AdaGossip 是一种新的技术，根据相邻代理之间的压缩模型差异自适应调整共识步长，通过大量实验证明在分布式学习中具有压缩通信时，相比于当前的最先进方法，在测试准确率上实现了卓越的性能（提高了 0-2% 的测试准确率）。