We study the problem of smooth imitation learning for online sequence prediction, where the goal is to train a policy that can smoothly imitate demonstrated behavior in a dynamic and continuous environment in response to online, sequential context input. Since the mapping from context to behavior is often complex, we take a learning reduction approach to reduce smooth imitation learning to a regression problem using complex function classes that are regularized to ensure smoothness. We present a learning meta-algorithm that achieves fast and stable convergence to a good policy. Our approach enjoys several attractive properties, including being fully deterministic, employing an adaptive learning rate that can provably yield larger policy improvements compared to previous approaches, and the ability to ensure stable convergence. Our empirical results demonstrate significant performance gains over previous approaches.

研究在线序列预测的平滑模仿学习问题，通过学习缩减的方法将其降低到回归问题，并采用正则化复杂函数类以确保平滑度。提出了一种元算法，实现快速稳定地收敛到好的策略，相比于以前的方法，具有全部确定性、自适应学习率等优点，并能保证稳定收敛，实证结果证明了性能得到了显著的提高。