Brain inspired spiking neural networks (SNNs) have been successfully applied
to many pattern recognition domains. The SNNs based deep structure have
achieved considerable results in perceptual tasks, such as image
classification, target detection. However, the application of deep SNNs in
reinforcement learning (RL) tasks is still a problem to be explored. Although
there have been previous studies on the combination of SNNs and RL, most of
them focus on robotic control problems with shallow networks or using ANN-SNN
conversion method to implement spiking deep Q Network (SDQN). In this work, we
mathematically analyzed the problem of the disappearance of spiking signal
features in SDQN and proposed a potential based layer normalization(pbLN)
method to directly train spiking deep Q networks. Experiment shows that
compared with state-of-art ANN-SNN conversion method and other SDQN works, the
proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on
Atari game tasks.

本文研究了基于脑启发的脉冲神经网络在强化学习任务中的应用问题，提出了可能基于层规范化方法直接训练脉冲神经网络。实验结果表明，相比于现有的基于人工神经网络和脉冲神经网络转换的方法，在 Atari 游戏任务中，所提出的可能基于层规范化的脉冲深度 Q 网络获得了更好的性能。