We address the issue of tuning hyperparameters (HPs) for imitation learning algorithms in the context of continuous-control, when the underlying reward function of the demonstrating expert cannot be observed at any time. The vast literature in imitation learning mostly considers this reward function to be available for HP selection, but this is not a realistic setting. Indeed, would this reward function be available, it could then directly be used for policy training and imitation would not be necessary. To tackle this mostly ignored problem, we propose a number of possible proxies to the external reward. We evaluate them in an extensive empirical study (more than 10'000 agents across 9 environments) and make practical recommendations for selecting HPs. Our results show that while imitation learning algorithms are sensitive to HP choices, it is often possible to select good enough HPs through a proxy to the reward function.

解决了连续控制环境的模仿学习算法中超参数调整时无法观测到专家演示的外部奖励函数的问题，并通过大量的实证研究提出多种可能的外部奖励代理以及选择超参数的实用建议，结果表明，虽然模仿学习算法对超参数选择敏感，但往往可以通过代理来选择足够好的超参数。

模仿学习的超参数选择