In adversarial imitation learning, a discriminator is trained to
differentiate agent episodes from expert demonstrations representing the
desired behavior. However, as the trained policy learns to be more successful,
the negative examples (the ones produced by the agent) become increasingly
similar to expert ones. Despite the fact that the task is successfully
accomplished in some of the agent's trajectories, the discriminator is trained
to output low values for them. We hypothesize that this inconsistent training
signal for the discriminator can impede its learning, and consequently leads to
worse overall performance of the agent. We show experimental evidence for this
hypothesis and that the 'False Negatives' (i.e. successful agent episodes)
significantly hinder adversarial imitation learning, which is the first
contribution of this paper. Then, we propose a method to alleviate the impact
of false negatives and test it on the BabyAI environment. This method
consistently improves sample efficiency over the baselines by at least an order
of magnitude.

通过对 Adversarial Imitation Learning 的训练信号做出调整，可以通过减少 False Negatives 的数量，从而在 BabyAI 环境中提高模型的样本效率。