With the rapid development of artificial intelligence technology, the field
of reinforcement learning has continuously achieved breakthroughs in both
theory and practice. However, traditional reinforcement learning algorithms
often entail high energy consumption during interactions with the environment.
Spiking Neural Network (SNN), with their low energy consumption characteristics
and performance comparable to deep neural networks, have garnered widespread
attention. To reduce the energy consumption of practical applications of
reinforcement learning, researchers have successively proposed the Pop-SAN and
MDC-SAN algorithms. Nonetheless, these algorithms use rectangular functions to
approximate the spike network during the training process, resulting in low
sensitivity, thus indicating room for improvement in the training effectiveness
of SNN. Based on this, we propose a trapezoidal approximation gradient method
to replace the spike network, which not only preserves the original stable
learning state but also enhances the model's adaptability and response
sensitivity under various signal dynamics. Simulation results show that the
improved algorithm, using the trapezoidal approximation gradient to replace the
spike network, achieves better convergence speed and performance compared to
the original algorithm and demonstrates good training stability.

基于梯形逼近梯度方法替代脉冲网络的改进算法在能耗、稳定学习状态、模型适应性和响应灵敏性等方面得出了较好的结果，相较于原算法具有更好的收敛速度和性能。