Distributed machine learning (DML) in mobile environments faces significant
communication bottlenecks. Gradient compression has emerged as an effective
solution to this issue, offering substantial benefits in environments with
limited bandwidth and metered data. Yet, they encounter severe performance drop
in non-IID environments due to a one-size-fits-all compression approach, which
does not account for the varying data volumes across workers. Assigning varying
compression ratios to workers with distinct data distributions and volumes is
thus a promising solution. This study introduces an analysis of distributed SGD
with non-uniform compression, which reveals that the convergence rate
(indicative of the iterations needed to achieve a certain accuracy) is
influenced by compression ratios applied to workers with differing volumes.
Accordingly, we frame relative compression ratio assignment as an $n$-variables
chi-square nonlinear optimization problem, constrained by a fixed and limited
communication budget. We propose DAGC-R, which assigns the worker handling
larger data volumes the conservative compression. Recognizing the computational
limitations of mobile devices, we DAGC-A, which are computationally less
demanding and enhances the robustness of the absolute gradient compressor in
non-IID scenarios. Our experiments confirm that both the DAGC-A and DAGC-R can
achieve better performance when dealing with highly imbalanced data volume
distribution and restricted communication.

在移动环境中，分布式机器学习面临着严重的通信瓶颈问题。梯度压缩已成为解决此问题的有效方法，在带宽有限和流量计费等环境中可以提供显著的好处。然而，在非 IID 环境中，由于一种适用于所有数据的压缩方法，它们遭遇到严重的性能下降。为具有不同数据分布和数据量的工作节点分配不同的压缩率因此是一个有前景的解决方案。本研究介绍了一种使用非均匀压缩的分布式 SGD 分析方法，揭示了收敛速率（指实现某个准确度所需的迭代次数）受应用于具有不同数据量的节点的压缩率的影响。因此，我们将相对压缩率分配作为一个 $n$ 个变量卡方非线性优化问题进行建模，并在通信开销有限的约束下进行求解。我们提出了 DAGC-R 方法，将具有较大数据量的节点分配保守的压缩率。鉴于移动设备的计算能力有限，我们还提出了 DAGC-A 方法，在非均匀数据分布的场景中具有较低的计算需求，并提高了绝对梯度压缩器的鲁棒性。实验证明，DAGC-A 和 DAGC-R 在处理高度不平衡的数据量分布和受限通信时能够获得更好的性能。