Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in both theory and often in practice. Some recent approaches provide stronger performance guarantees in this setting, but remain somewhat unsatisfactory as they train either non-stationary or a stochastic policies and require a large number of iterations. In this paper, we propose a new iterative algorithm, which trains a stationary deterministic policy, that can be seen as a no regret algorithm in an online learning setting. We show that any such no regret algorithm, combined with additional reduction assumptions, must find a policy with good performance under the distribution of observations it induces in such sequential settings. We additionally show that this new approach outperforms previous approaches on two challenging imitation learning problem and a benchmark sequence labeling problem.

本文提出了一种新的迭代算法，该算法在在线学习环境中训练一个稳定的确定性策略，结合特定的降维假设，找到了具有良好性能的策略，克服了之前方法的一些不足，实验表明该方法在两个挑战性的仿真学习问题和基准序列标记问题上表现优异。

将模仿学习和结构化预测转化为无遗憾在线学习