Training robotic policies in simulation suffers from the sim-to-real gap, as
simulated dynamics can be different from real-world dynamics. Past works
tackled this problem through domain randomization and online
system-identification. The former is sensitive to the manually-specified
training distribution of dynamics parameters and can result in behaviors that
are overly conservative. The latter requires learning policies that
concurrently perform the task and generate useful trajectories for system
identification. In this work, we propose and analyze a framework for learning
exploration policies that explicitly perform task-oriented exploration actions
to identify task-relevant system parameters. These parameters are then used by
model-based trajectory optimization algorithms to perform the task in the real
world. We instantiate the framework in simulation with the Linear Quadratic
Regulator as well as in the real world with pouring and object dragging tasks.
Experiments show that task-oriented exploration helps model-based policies
adapt to systems with initially unknown parameters, and it leads to better task
performance than task-agnostic exploration.

本文提出并分析了一种基于任务导向探索的框架，该框架通过在模拟环境中学习任务导向的探索策略来标识任务相关的系统参数，并将这些参数用于在现实世界中进行基于模型的轨迹优化。实验表明，任务导向的探索有助于在系统参数未知时，使基于模型的策略适应更好，从而实现更好的任务表现。