Model-based reasoning agents are ill-equipped to act in novel situations in
which their model of the environment no longer sufficiently represents the
world. We propose HYDRA - a framework for designing model-based agents
operating in mixed discrete-continuous worlds, that can autonomously detect
when the environment has evolved from its canonical setup, understand how it
has evolved, and adapt the agents' models to perform effectively. HYDRA is
based upon PDDL+, a rich modeling language for planning in mixed,
discrete-continuous environments. It augments the planning module with visual
reasoning, task selection, and action execution modules for closed-loop
interaction with complex environments. HYDRA implements a novel meta-reasoning
process that enables the agent to monitor its own behavior from a variety of
aspects. The process employs a diverse set of computational methods to maintain
expectations about the agent's own behavior in an environment. Divergences from
those expectations are useful in detecting when the environment has evolved and
identifying opportunities to adapt the underlying models. HYDRA builds upon
ideas from diagnosis and repair and uses a heuristics-guided search over model
changes such that they become competent in novel conditions. The HYDRA
framework has been used to implement novelty-aware agents for three diverse
domains - CartPole++ (a higher dimension variant of a classic control problem),
Science Birds (an IJCAI competition problem), and PogoStick (a specific problem
domain in Minecraft). We report empirical observations from these domains to
demonstrate the efficacy of various components in the novelty meta-reasoning
process.

提出了一种名为 HYDRA 的框架，通过引入视觉推理、任务选择和行动执行模块，以及一种新颖的元推理过程，使模型引导的 agent 能够在混合离散 - 连续环境中自动检测、理解并适应其演化，实现对复杂环境的闭环交互。