Spiking neural networks (SNNs) are widely applied in various fields due to
their energy-efficient and fast-inference capabilities. Applying SNNs to
reinforcement learning (RL) can significantly reduce the computational resource
requirements for agents and improve the algorithm's performance under
resource-constrained conditions. However, in current spiking reinforcement
learning (SRL) algorithms, the simulation results of multiple time steps can
only correspond to a single-step decision in RL. This is quite different from
the real temporal dynamics in the brain and also fails to fully exploit the
capacity of SNNs to process temporal data. In order to address this temporal
mismatch issue and further take advantage of the inherent temporal dynamics of
spiking neurons, we propose a novel temporal alignment paradigm (TAP) that
leverages the single-step update of spiking neurons to accumulate historical
state information in RL and introduces gated units to enhance the memory
capacity of spiking neurons. Experimental results show that our method can
solve partially observable Markov decision processes (POMDPs) and multi-agent
cooperation problems with similar performance as recurrent neural networks
(RNNs) but with about 50% power consumption.

利用时间对齐范式 (TAP) 解决当前脉冲神经网络在增强学习中存在的时间不匹配问题，提高处理时间数据的能力，能在与递归神经网络 (RNNs) 相似性能下降低 50% 的电力消耗下解决部分可观测马尔科夫决策过程 (POMDPs) 和多智能体合作问题。