The use of sparse neural networks has seen rapid growth in recent years, particularly in computer vision. Their appeal stems largely from the reduced number of parameters required to train and store, as well as in an increase in learning efficiency. Somewhat surprisingly, there have been very few efforts exploring their use in Deep Reinforcement Learning (DRL). In this work we perform a systematic investigation into applying a number of existing sparse training techniques on a variety of DRL agents and environments. Our results corroborate the findings from sparse training in the computer vision domain - sparse networks perform better than dense networks for the same parameter count - in the DRL domain. We provide detailed analyses on how the various components in DRL are affected by the use of sparse networks and conclude by suggesting promising avenues for improving the effectiveness of sparse training methods, as well as for advancing their use in DRL.

在深度强化学习领域，这项工作系统地研究了应用多种现有的稀疏训练技术在各种强化学习代理和环境中的可行性，结果发现，稀疏网络比密集网络在相同数量参数下表现更好，我们提供了有关如何改善稀疏训练方法有效性以及推进其在深度强化学习中应用的有益思路。

深度强化学习稀疏训练的现状