Many practically relevant robot grasping problems feature a target object for
which all grasps are occluded, e.g., by the environment. Single-shot grasp
planning invariably fails in such scenarios. Instead, it is necessary to first
manipulate the object into a configuration that affords a grasp. We solve this
problem by learning a sequence of actions that utilize the environment to
change the object's pose. Concretely, we employ hierarchical reinforcement
learning to combine a sequence of learned parameterized manipulation
primitives. By learning the low-level manipulation policies, our approach can
control the object's state through exploiting interactions between the object,
the gripper, and the environment. Designing such a complex behavior
analytically would be infeasible under uncontrolled conditions, as an analytic
approach requires accurate physical modeling of the interaction and contact
dynamics. In contrast, we learn a hierarchical policy model that operates
directly on depth perception data, without the need for object detection, pose
estimation, or manual design of controllers. We evaluate our approach on
picking box-shaped objects of various weight, shape, and friction properties
from a constrained table-top workspace. Our method transfers to a real robot
and is able to successfully complete the object picking task in 98\% of
experimental trials.

通过学习层次化强化学习，我们能够利用环境改变目标物体的姿态，无需物体检测、姿态估计或手动设计控制器，从而成功地完成 98% 的实验任务。