As a general method for exploration in deep reinforcement learning (RL),
NoisyNet can produce problem-specific exploration strategies. Spiking neural
networks (SNNs), due to their binary firing mechanism, have strong robustness
to noise, making it difficult to realize efficient exploration with local
disturbances. To solve this exploration problem, we propose a noisy spiking
actor network (NoisySAN) that introduces time-correlated noise during charging
and transmission. Moreover, a noise reduction method is proposed to find a
stable policy for the agent. Extensive experimental results demonstrate that
our method outperforms the state-of-the-art performance on a wide range of
continuous control tasks from OpenAI gym.

通过引入时序相关的噪音，我们提出了一种噪声尖峰神经体执行者网络（NoisySAN）来解决深度强化学习中的探索问题，并提出了一种噪音减少方法，以找到智能体的稳定策略。大量实验结果表明，我们的方法在 OpenAI gym 的各种连续控制任务中表现优于现有技术水平。

噪声的尖峰演员网络探索

Noisy Spiking Actor Network for Exploration

We introduce NoisyNet, a deep reinforcement learning agent with parametric
noise added to its weights, and show that the induced stochasticity of the
agent's policy can be used to aid efficient exploration. The parameters of the
noise are learned with gradient descent along with the remaining network
weights. NoisyNet is straightforward to implement and adds little computational
overhead. We find that replacing the conventional exploration heuristics for
A3C, DQN and dueling agents (entropy reward and $\epsilon$-greedy respectively)
with NoisyNet yields substantially higher scores for a wide range of Atari
games, in some cases advancing the agent from sub to super-human performance.

本研究提出了一种名为 NoisyNet 的深度强化学习智能体，通过参数噪声的添加，可以有效地探索问题空间，在 Atari 游戏中，用 NoisyNet 替换传统的探索启发式方法后，智能体的分数得到了大幅提升，有些情况下，甚至可以达到超人类水平。