Prior approximations of AIXI, a Bayesian optimality notion for general
reinforcement learning, can only approximate AIXI's Bayesian environment model
using an a-priori defined set of models. This is a fundamental source of
epistemic uncertainty for the agent in settings where the existence of
systematic bias in the predefined model class cannot be resolved by simply
collecting more data from the environment. We address this issue in the context
of Human-AI teaming by considering a setup where additional knowledge for the
agent in the form of new candidate models arrives from a human operator in an
online fashion. We introduce a new agent called DynamicHedgeAIXI that maintains
an exact Bayesian mixture over dynamically changing sets of models via a
time-adaptive prior constructed from a variant of the Hedge algorithm. The
DynamicHedgeAIXI agent is the richest direct approximation of AIXI known to
date and comes with good performance guarantees. Experimental results on
epidemic control on contact networks validates the agent's practical utility.

通过引入一个名为 DynamicHedgeAIXI 的新代理实现对 AIXI 的贴切逼近，通过使用一种时间自适应优先级构建的贝叶斯混合模型，该代理在表示动态变化的模型集合时表现出良好性能，实验证实该代理在传染病控制方面的实际效用。