Sample efficiency is central to developing practical reinforcement learning
(RL) for complex and large-scale decision-making problems. The ability to
transfer and generalize knowledge gained from previous experiences to
downstream tasks can significantly improve sample efficiency. Recent research
indicates that successor feature (SF) RL algorithms enable knowledge
generalization between tasks with different rewards but identical transition
dynamics. It has recently been hypothesized that combining model-based (MB)
methods with SF algorithms can alleviate the limitation of fixed transition
dynamics. Furthermore, uncertainty-aware exploration is widely recognized as
another appealing approach for improving sample efficiency. Putting together
two ideas of hybrid model-based successor feature (MB-SF) and uncertainty leads
to an approach to the problem of sample efficient uncertainty-aware knowledge
transfer across tasks with different transition dynamics or/and reward
functions. In this paper, the uncertainty of the value of each action is
approximated by a Kalman filter (KF)-based multiple-model adaptive estimation.
This KF-based framework treats the parameters of a model as random variables.
To the best of our knowledge, this is the first attempt at formulating a hybrid
MB-SF algorithm capable of generalizing knowledge across large or continuous
state space tasks with various transition dynamics while requiring less
computation at decision time than MB methods. The number of samples required to
learn the tasks was compared to recent SF and MB baselines. The results show
that our algorithm generalizes its knowledge across different transition
dynamics, learns downstream tasks with significantly fewer samples than
starting from scratch, and outperforms existing approaches.

使用混合模型驱动的继任特征算法和不确定性感知探索的方法，能够在具有不同转换动力学或 / 和奖励函数的任务之间实现高效的知识传输，并且在决策时间上所需的计算量较少。通过与最近的继任特征算法和模型驱动方法进行对比，结果表明我们的算法能够在不同的转换动力学中泛化知识，使用明显更少的样本学习下游任务，并且优于现有方法。