In lifelong learning, an agent learns throughout its entire life without
resets, in a constantly changing environment, as we humans do. Consequently,
lifelong learning comes with a plethora of research problems such as continual
domain shifts, which result in non-stationary rewards and environment dynamics.
These non-stationarities are difficult to detect and cope with due to their
continuous nature. Therefore, exploration strategies and learning methods are
required that are capable of tracking the steady domain shifts, and adapting to
them. We propose Reactive Exploration to track and react to continual domain
shifts in lifelong reinforcement learning, and to update the policy
correspondingly. To this end, we conduct experiments in order to investigate
different exploration strategies. We empirically show that representatives of
the policy-gradient family are better suited for lifelong learning, as they
adapt more quickly to distribution shifts than Q-learning. Thereby,
policy-gradient methods profit the most from Reactive Exploration and show good
results in lifelong learning with continual domain shifts. Our code is
available at: this https URL.

该论文探讨了在终身学习的情境下如何跟踪和适应持续的领域转变，为此提出了反应式探索方法，并基于实验证明，策略梯度方法是适合于终身学习的一种学习方法，能更快地适应分布变化。