Deep reinforcement learning (DRL) has shown remarkable success in sequential
decision-making problems but suffers from a long training time to obtain such
good performance. Many parallel and distributed DRL training approaches have
been proposed to solve this problem, but it is difficult to utilize them on
resource-limited devices. In order to accelerate DRL in real-world edge
devices, memory bandwidth bottlenecks due to large weight transactions have to
be resolved. However, previous iterative pruning not only shows a low
compression ratio at the beginning of training but also makes DRL training
unstable. To overcome these shortcomings, we propose a novel weight compression
method for DRL training acceleration, named group-sparse training (GST). GST
selectively utilizes block-circulant compression to maintain a high weight
compression ratio during all iterations of DRL training and dynamically adapt
target sparsity through reward-aware pruning for stable training. Thanks to the
features, GST achieves a 25 \%p $\sim$ 41.5 \%p higher average compression
ratio than the iterative pruning method without reward drop in Mujoco
Halfcheetah-v2 and Mujoco humanoid-v2 environment with TD3 training.

为了加速深度强化学习在实际边缘设备中的运行，本文提出了一种基于分块循环压缩的权重压缩方法，名为组稀疏训练 (GST)，该方法在所有深度强化学习迭代中选择性地使用分块循环压缩，并通过奖励感知修剪动态地调整目标稀疏度，以实现稳定的训练，实验表明该方法在 Mujoco Halfcheetah-v2 和 Mujoco humanoid-v2 环境下的 TD3 训练中，平均压缩比比迭代修剪方法高 25% p-41.5% p，而且没有奖励降低。