Current AI systems are designed to solve close-world problems with the
assumption that the underlying world is remaining more or less the same.
However, when dealing with real-world problems such assumptions can be invalid
as sudden and unexpected changes can occur. To effectively deploy AI-powered
systems in the real world, AI systems should be able to deal with open-world
novelty quickly. Inevitably, dealing with open-world novelty raises an
important question of novelty difficulty. Knowing whether one novelty is harder
to deal with than another, can help researchers to train their systems
systematically. In addition, it can also serve as a measurement of the
performance of novelty robust AI systems. In this paper, we propose to define
the novelty reaction difficulty as a relative difficulty of performing the
known task after the introduction of the novelty. We propose a universal method
that can be applied to approximate the difficulty. We present the
approximations of the difficulty using our method and show how it aligns with
the results of the evaluation of AI agents designed to deal with novelty.

本文提出一种评估 AI 系统应对开放世界新颖问题难度的方法，以帮助研究人员系统地训练其系统，并衡量其表现。

测量新颖反应的难度

Measuring Difficulty of Novelty Reaction

Open-world novelty occurs when the rules of an environment can change
abruptly, such as when a game player encounters "house rules". To address
open-world novelty, game playing agents must be able to detect when novelty is
injected, and to quickly adapt to the new rules. We propose a model-based
reinforcement learning approach where game state and rules are represented as
knowledge graphs. The knowledge graph representation of the state and rules
allows novelty to be detected as changes in the knowledge graph, assists with
the training of deep reinforcement learners, and enables imagination-based
re-training where the agent uses the knowledge graph to perform look-ahead.

该研究提出了一种基于模型的强化学习方法，在该方法中，游戏状态和规则都是以知识图谱的形式表示，有助于检测游戏环境中的开放性创新，并进行智能重训练。