Non-parametric episodic memory can be used to quickly latch onto high-reward
experience in reinforcement learning tasks. In contrast to parametric deep
reinforcement learning approaches, these methods only need to discover the
solution once, and may then repeatedly solve the task. However, episodic
control solutions are stored in discrete tables, and this approach has so far
only been applied to discrete action space problems. Therefore, this paper
introduces Continuous Episodic Control (CEC), a novel non-parametric episodic
memory algorithm for sequential decision making in problems with a continuous
action space. Results on several sparse-reward continuous control environments
show that our proposed method learns faster than state-of-the-art model-free RL
and memory-augmented RL algorithms, while maintaining good long-run performance
as well. In short, CEC can be a fast approach for learning in continuous
control tasks, and a useful addition to parametric RL methods in a hybrid
approach as well.

本文提出了一种新型非参数迭代记忆算法 CEC 来解决具有连续动作空间的顺序决策问题，同时在多个稀疏奖励连续控制环境中表现出比最先进的无模型 RL 和记忆扩展 RL 算法更快的学习速度和更好的长期表现。