Inverse Reinforcement Learning (IRL) learns an optimal policy, given some
expert demonstrations, thus avoiding the need for the tedious process of
specifying a suitable reward function. However, current methods are constrained
by at least one of the following requirements. The first one is the need to
fully solve a forward Reinforcement Learning (RL) problem in the inner loop of
the algorithm, which might be prohibitively expensive in many complex
environments. The second one is the need for full trajectories from the
experts, which might not be easily available. The third one is the assumption
that the expert data is homogeneous rather than a collection from various
experts or possibly alternative solutions to the same task. Such constraints
make IRL approaches either not scalable or not usable on certain existing
systems. In this work we propose an approach which removes these requirements
through a dynamic, adaptive method called Adversarial Imitation Learning on
Aggregated Data (AILAD). It learns conjointly both a non linear reward function
and the associated optimal policy using an adversarial framework. The reward
learner only uses aggregated data. Moreover, it generates diverse behaviors
producing a distribution over the aggregated data matching that of the experts.

通过使用聚合数据上的对抗性模仿学习，我们提出了一种名为 Adversarial Imitation Learning on Aggregated Data (AILAD) 的动态自适应方法，可以同时学习非线性奖励函数和相关的最优策略，并且生成多样化的行为来匹配专家数据的分布。