Recent work on deep reinforcement learning (DRL) has pointed out that
algorithmic information about good policies can be extracted from offline data
which lack explicit information about executed actions. For example, videos of
humans or robots may convey a lot of implicit information about rewarding
action sequences, but a DRL machine that wants to profit from watching such
videos must first learn by itself to identify and recognize relevant
states/actions/rewards. Without relying on ground-truth annotations, our new
method called Deep State Identifier learns to predict returns from episodes
encoded as videos. Then it uses a kind of mask-based sensitivity analysis to
extract/identify important critical states. Extensive experiments showcase our
method's potential for understanding and improving agent behavior. The source
code and the generated datasets are available at
this https URL

最近的深度强化学习研究指出，即使缺乏关于执行动作的显式信息，也可以从离线数据中提取有关良好策略的算法信息。本文介绍一种名为 Deep State Identifier 的新方法，该方法通过编码为视频的剧集学习预测回报，并利用一种基于掩码的敏感性分析来提取和识别重要的关键状态。大量实验证明了我们方法理解和改进代理行为的潜力。源代码和生成的数据集可在此链接中获得。