A robust body of reinforcement learning techniques have been developed to
solve complex sequential decision making problems. However, these methods
assume that train and evaluation tasks come from similarly or identically
distributed environments. This assumption does not hold in real life where
small novel changes to the environment can make a previously learned policy
fail or introduce simpler solutions that might never be found. To that end we
explore the concept of {\em novelty}, defined in this work as the sudden change
to the mechanics or properties of environment. We provide an ontology of for
novelties most relevant to sequential decision making, which distinguishes
between novelties that affect objects versus actions, unary properties versus
non-unary relations, and the distribution of solutions to a task. We introduce
NovGrid, a novelty generation framework built on MiniGrid, acting as a toolkit
for rapidly developing and evaluating novelty-adaptation-enabled reinforcement
learning techniques. Along with the core NovGrid we provide exemplar novelties
aligned with our ontology and instantiate them as novelty templates that can be
applied to many MiniGrid-compliant environments. Finally, we present a set of
metrics built into our framework for the evaluation of
novelty-adaptation-enabled machine-learning techniques, and show
characteristics of a baseline RL model using these metrics.

研究提出了一种新奇性改进的强化学习方法，针对具有改变环境机制 / 属性的新奇性，提出了 NovGrid 框架及其本体，以及可以应用于 MiniGrid 兼容环境的新奇性模板，并通过内置度量指标评估了基线强化学习模型的特征。