Model-free policy learning has been shown to be capable of learning
manipulation policies which can solve long-time horizon tasks using single-step
manipulation primitives. However, training these policies is a time-consuming
process requiring large amounts of data. We propose the Local Dynamics Model
(LDM) which efficiently learns the state-transition function for these
manipulation primitives. By combining the LDM with model-free policy learning,
we can learn policies which can solve complex manipulation tasks using one-step
lookahead planning. We show that the LDM is both more sample-efficient and
outperforms other model architectures. When combined with planning, we can
outperform other model-based and model-free policies on several challenging
manipulation tasks in simulation.

本文介绍了一种通过 Local Dynamics Model 和无模型策略学习相结合的方式有效地学习状态转移函数从而能够解决一步先见规划的复杂操纵任务的方法，并在模拟中展示了本方法的优越性。

使用本地动态模型的视觉预测

Visual Foresight With a Local Dynamics Model

There is a widespread intuition that model-based control methods should be
able to surpass the data efficiency of model-free approaches. In this paper we
attempt to evaluate this intuition on various challenging locomotion tasks. We
take a hybrid approach, combining model predictive control (MPC) with a learned
model and model-free policy learning; the learned policy serves as a proposal
for MPC. We find that well-tuned model-free agents are strong baselines even
for high DoF control problems but MPC with learned proposals and models
(trained on the fly or transferred from related tasks) can significantly
improve performance and data efficiency in hard multi-task/multi-goal settings.
Finally, we show that it is possible to distil a model-based planner into a
policy that amortizes the planning computation without any loss of performance.
Videos of agents performing different tasks can be seen at
this https URL

本文探讨了基于模型的控制方法是否能够超越基于数据的方法。研究人员通过将模型预测控制与学习模型和基于数据的策略学习相结合的方式，对多种具有挑战性的运动任务进行了评估，并发现：经过良好调节的基于数据的策略学习代理是高自由度控制问题的强基准。但是，针对困难的多任务 / 多目标场景，经过训练的动态模型和学习的策略作为模型预测控制的建议，可以显著提高性能和数据效率。最后，研究表明，即使没有性能损失，也可以将基于模型的规划器简化为一种策略，从而将计划计算的负担分担到了策略中。