Reinforcement learning (RL) faces substantial challenges when applied to
real-life problems, primarily stemming from the scarcity of available data due
to limited interactions with the environment. This limitation is exacerbated by
the fact that RL often demands a considerable volume of data for effective
learning. The complexity escalates further when implementing RL in recurrent
spiking networks, where inherent noise introduced by spikes adds a layer of
difficulty. Life-long learning machines must inherently resolve the
plasticity-stability paradox. Striking a balance between acquiring new
knowledge and maintaining stability is crucial for artificial agents. In this
context, we take inspiration from machine learning technology and introduce a
biologically plausible implementation of proximal policy optimization, arguing
that it significantly alleviates this challenge. Our approach yields two
notable advancements: first, the ability to assimilate new information without
necessitating alterations to the current policy, and second, the capability to
replay experiences without succumbing to policy divergence. Furthermore, when
contrasted with other experience replay (ER) techniques, our method
demonstrates the added advantage of being computationally efficient in an
online setting. We demonstrate that the proposed methodology enhances the
efficiency of learning, showcasing its potential impact on neuromorphic and
real-world applications.

强化学习面临着应用于现实问题的巨大挑战，主要源于有限的与环境交互导致的可用数据的稀缺性。本研究引入了生物学上可行的近端策略优化的实现，通过在重要领域中显著减轻这一挑战，提高了学习的效率。