We present the problem of reinforcement learning with exogenous termination.
We define the Termination Markov Decision Process (TerMDP), an extension of the
MDP framework, in which episodes may be interrupted by an external
non-Markovian observer. This formulation accounts for numerous real-world
situations, such as a human interrupting an autonomous driving agent for
reasons of discomfort. We learn the parameters of the TerMDP and leverage the
structure of the estimation problem to provide state-wise confidence bounds. We
use these to construct a provably-efficient algorithm, which accounts for
termination, and bound its regret. Motivated by our theoretical analysis, we
design and implement a scalable approach, which combines optimism (w.r.t.
termination) and a dynamic discount factor, incorporating the termination
probability. We deploy our method on high-dimensional driving and MinAtar
benchmarks. Additionally, we test our approach on human data in a driving
setting. Our results demonstrate fast convergence and significant improvement
over various baseline approaches.

文章探讨了外部中断对强化学习的影响，提出了 TerMDP 框架并应用于驾驶和 MinAtar 测试中，提出了一种基于动态折扣因子和置信区间的高效算法，并与多种基准方法相比较获得了显著提升