The Option Keyboard (OK) was recently proposed as a method for transferring
behavioral knowledge across tasks. OK transfers knowledge by adaptively
combining subsets of known behaviors using Successor Features (SFs) and
Generalized Policy Improvement (GPI). However, it relies on hand-designed
state-features and task encodings which are cumbersome to design for every new
environment. In this work, we propose the "Successor Features Keyboard" (SFK),
which enables transfer with discovered state-features and task encodings. To
enable discovery, we propose the "Categorical Successor Feature Approximator"
(CSFA), a novel learning algorithm for estimating SFs while jointly discovering
state-features and task encodings. With SFK and CSFA, we achieve the first
demonstration of transfer with SFs in a challenging 3D environment where all
the necessary representations are discovered. We first compare CSFA against
other methods for approximating SFs and show that only CSFA discovers
representations compatible with SF&GPI at this scale. We then compare SFK
against transfer learning baselines and show that it transfers most quickly to
long-horizon tasks.

使用最新提出的 “Option Keyboard”(OK) 方法，本研究提出了 “Successor Features Keyboard”(SFK) 和 “Categorical Successor Feature Approximator”(CSFA) 两种方法，实现在一个具有挑战性的三维环境中，通过发现状态特征和任务编码进行知识迁移。通过对比其他方法和基准的实验结果，结果显示 CSFA 能够以更高效的方式进行 SF 近似，并且 SFK 在长期任务中最快地进行知识迁移。