We introduce a novel apprenticeship learning algorithm to learn an expert's underlying reward structure in off-policy model-free \emph{batch} settings. Unlike existing methods that require a dynamics model or additional data acquisition for on-policy evaluation, our algorithm requires only the batch data of observed expert behavior. Such settings are common in real-world tasks---health care, finance or industrial processes ---where accurate simulators do not exist or data acquisition is costly. To address challenges in batch settings, we introduce Deep Successor Feature Networks(DSFN) that estimate feature expectations in an off-policy setting and a transition-regularized imitation network that produces a near-expert initial policy and an efficient feature representation. Our algorithm achieves superior results in batch settings on both control benchmarks and a vital clinical task of sepsis management in the Intensive Care Unit.

本篇论文介绍了一种新的学徒式学习算法，通过仅利用专家行为批量数据来学习专家的潜在奖励结构，并使用深继承特征网络(DSFN)来估计特征期望值和过渡规范化模仿网络来产生高效的特征表示和接近专家的初始政策，实现了对临床Intensive Care Unit疾病Septic Shock的卓越处理结果。

带深度继承特征的真正批量学徒学习