Deep reinforcement learning (RL) algorithms can learn complex robotic skills
from raw sensory inputs, but have yet to achieve the kind of broad
generalization and applicability demonstrated by deep learning methods in
supervised domains. We present a deep RL method that is practical for
real-world robotics tasks, such as robotic manipulation, and generalizes
effectively to never-before-seen tasks and objects. In these settings, ground
truth reward signals are typically unavailable, and we therefore propose a
self-supervised model-based approach, where a predictive model learns to
directly predict the future from raw sensory readings, such as camera images.
At test time, we explore three distinct goal specification methods: designated
pixels, where a user specifies desired object manipulation tasks by selecting
particular pixels in an image and corresponding goal positions, goal images,
where the desired goal state is specified with an image, and image classifiers,
which define spaces of goal states. Our deep predictive models are trained
using data collected autonomously and continuously by a robot interacting with
hundreds of objects, without human supervision. We demonstrate that visual MPC
can generalize to never-before-seen objects---both rigid and deformable---and
solve a range of user-defined object manipulation tasks using the same model.

本文提出了一种实用于实际机器人任务 -- 如机器人操作 -- 的深度强化学习算法，并在无人监督的情况下使用自我监督的基于模型的方法来训练预测模型，通过选择指定像素、目标图像或图像分类器作为目标设定方法，探索实现前所未见的任务和物体的普遍泛化。