Spiking neural networks (SNNs) have great potential for energy-efficient
implementation of Deep Neural Networks (DNNs) on dedicated neuromorphic
hardware. Recent studies demonstrated competitive performance of SNNs compared
with DNNs on image classification tasks, including CIFAR-10 and ImageNet data.
The present work focuses on using SNNs in combination with deep reinforcement
learning in ATARI games, which involves additional complexity as compared to
image classification. We review the theory of converting DNNs to SNNs and
extending the conversion to Deep Q-Networks (DQNs). We propose a robust
representation of the firing rate to reduce the error during the conversion
process. In addition, we introduce a new metric to evaluate the conversion
process by comparing the decisions made by the DQN and SNN, respectively. We
also analyze how the simulation time and parameter normalization influence the
performance of converted SNNs. We achieve competitive scores on 17
top-performing Atari games. To the best of our knowledge, our work is the first
to achieve state-of-the-art performance on multiple Atari games with SNNs. Our
work serves as a benchmark for the conversion of DQNs to SNNs and paves the way
for further research on solving reinforcement learning tasks with SNNs.

本研究旨在将 DNN 转化为 SNN，并在使用深度强化学习进行 ATARI 游戏时提高性能及结果。研究采用新的度量方法和模拟时间参数标准，最终在 17 种 ATARI 游戏中达到最优性能，为以后采用 SNNs 解决深度强化学习任务提供基准和导引。