Gradient compression alleviates expensive communication in distributed deep
learning by sending fewer values and its corresponding indices, typically via
Allgather (AG). Training with high compression ratio (CR) achieves high
accuracy like DenseSGD, but has lower parallel scaling due to high
communication cost (i.e., parallel efficiency). Using lower CRs improves
parallel efficiency by lowering synchronization cost, but degrades model
accuracy as well (statistical efficiency). Further, speedup attained with
different models and CRs also varies with network latency, effective bandwidth
and collective op used for aggregation. In many cases, collectives like
Allreduce (AR) have lower cost than AG to exchange the same amount of data. In
this paper, we propose an AR-compatible Topk compressor that is
bandwidth-optimal and thus performs better than AG in certain network
configurations. We develop a flexible communication strategy that switches
between AG and AR based on which collective is optimal in the current settings,
and model the pareto-relationship between parallel and statistical efficiency
as a multi-objective optimization (MOO) problem to dynamically adjust CR and
accelerate training while still converging to high accuracy.

梯度压缩通过发送更少的值和对应的索引（通常通过 Allgather）来减轻分布式深度学习中昂贵的通信成本。本文提出了一种与 Allreduce 兼容的 Topk 压缩器，在某些网络配置中比 Allgather 表现更好。我们开发了一种灵活的通信策略，根据当前设置中哪个集合操作最优，切换 AG 和 AR 之间的选择，并将并行效率和统计效率的帕累托关系建模为一个多目标优化问题，以动态调整压缩比并加速训练，同时仍收敛于高准确性。

灵活通信实现不确定网络下的最优分布式学习

Flexible Communication for Optimal Distributed Learning over  Unpredictable Networks

Deep reinforcement learning (DRL) algorithms and evolution strategies (ES)
have been applied to various tasks, showing excellent performances. These have
the opposite properties, with DRL having good sample efficiency and poor
stability, while ES being vice versa. Recently, there have been attempts to
combine these algorithms, but these methods fully rely on synchronous update
scheme, making it not ideal to maximize the benefits of the parallelism in ES.
To solve this challenge, asynchronous update scheme was introduced, which is
capable of good time-efficiency and diverse policy exploration. In this paper,
we introduce an Asynchronous Evolution Strategy-Reinforcement Learning (AES-RL)
that maximizes the parallel efficiency of ES and integrates it with policy
gradient methods. Specifically, we propose 1) a novel framework to merge ES and
DRL asynchronously and 2) various asynchronous update methods that can take all
advantages of asynchronism, ES, and DRL, which are exploration and time
efficiency, stability, and sample efficiency, respectively. The proposed
framework and update methods are evaluated in continuous control benchmark
work, showing superior performance as well as time efficiency compared to the
previous methods.

本研究提出了一种异步进化策略强化学习 (AES-RL) 算法，将进化策略和策略梯度结合，并引入异步更新的方法，能够在连续控制基准测试中表现出优异的性能和时间效率。