A major challenge in robotics is to design robust policies which enable
complex and agile behaviors in the real world. On one end of the spectrum, we
have model-free reinforcement learning (MFRL), which is incredibly flexible and
general but often results in brittle policies. In contrast, model predictive
control (MPC) continually re-plans at each time step to remain robust to
perturbations and model inaccuracies. However, despite its real-world
successes, MPC often under-performs the optimal strategy. This is due to model
quality, myopic behavior from short planning horizons, and approximations due
to computational constraints. And even with a perfect model and enough compute,
MPC can get stuck in bad local optima, depending heavily on the quality of the
optimization algorithm. To this end, we propose Deep Model Predictive
Optimization (DMPO), which learns the inner-loop of an MPC optimization
algorithm directly via experience, specifically tailored to the needs of the
control problem. We evaluate DMPO on a real quadrotor agile trajectory tracking
task, on which it improves performance over a baseline MPC algorithm for a
given computational budget. It can outperform the best MPC algorithm by up to
27% with fewer samples and an end-to-end policy trained with MFRL by 19%.
Moreover, because DMPO requires fewer samples, it can also achieve these
benefits with 4.3X less memory. When we subject the quadrotor to turbulent wind
fields with an attached drag plate, DMPO can adapt zero-shot while still
outperforming all baselines. Additional results can be found at
this https URL

机器人学中的一个主要挑战是设计出能在现实世界中实现复杂和灵活行为的稳健策略。我们提出了一种名为 “Deep Model Predictive Optimization” 的方法，通过经验直接学习 MPC 优化算法的内环，针对控制问题的需求进行特定优化。通过在一个真实的四旋翼平衡轨迹跟踪任务中的评估，DMPO 在计算预算下提高了性能，并且比基线的 MPC 算法在样本数量和端到端策略训练（MFRL）方面分别提高了 27% 和 19%。此外，由于 DMPO 需要更少的样本，它还可以以 4.3 倍的内存减少实现这些效益。当我们将四旋翼暴露在带有阻力板的扰动风场中时，DMPO 能够零 - shot 调适，同时仍然优于所有基线测试结果。