We focus on the task of goal-oriented grasping, in which a robot is supposed
to grasp a pre-assigned goal object in clutter and needs some pre-grasp actions
such as pushes to enable stable grasps. However, in this task, the robot gets
positive rewards from environment only when successfully grasping the goal
object. Besides, joint pushing and grasping elongates the action sequence,
compounding the problem of reward delay. Thus, sample inefficiency remains a
main challenge in this task. In this paper, a goal-conditioned hierarchical
reinforcement learning formulation with high sample efficiency is proposed to
learn a push-grasping policy for grasping a specific object in clutter. In our
work, sample efficiency is improved by two means. First, we use a
goal-conditioned mechanism by goal relabeling to enrich the replay buffer.
Second, the pushing and grasping policies are respectively regarded as a
generator and a discriminator and the pushing policy is trained with
supervision of the grasping discriminator, thus densifying pushing rewards. To
deal with the problem of distribution mismatch caused by different training
settings of two policies, an alternating training stage is added to learn
pushing and grasping in turn. A series of experiments carried out in simulation
and real world indicate that our method can quickly learn effective pushing and
grasping policies and outperforms existing methods in task completion rate and
goal grasp success rate by less times of motion. Furthermore, we validate that
our system can also adapt to goal-agnostic conditions with better performance.
Note that our system can be transferred to the real world without any
fine-tuning. Our code is available at
this https URL

本文提出了一种高样本效率的目标导向层次强化学习模型，使用推和抓策略来实现在混乱环境下抓取指定目标物体，经过一系列实验验证表明，该模型表现出了高的任务完成率和目标抓取成功率，并能够适应目标不明确的条件，并且可以直接转移到实际应用中。

在混乱环境中有效学习面向目标的推 - 抓协同技能

Efficient learning of goal-oriented push-grasping synergy in clutter

Robots have to face challenging perceptual settings, including changes in
viewpoint, lighting, and background. Current simulated reinforcement learning
(RL) benchmarks such as DM Control provide visual input without such
complexity, which limits the transfer of well-performing methods to the real
world. In this paper, we extend DM Control with three kinds of visual
distractions (variations in background, color, and camera pose) to produce a
new challenging benchmark for vision-based control, and we analyze state of the
art RL algorithms in these settings. Our experiments show that current RL
methods for vision-based control perform poorly under distractions, and that
their performance decreases with increasing distraction complexity, showing
that new methods are needed to cope with the visual complexities of the real
world. We also find that combinations of multiple distraction types are more
difficult than a mere combination of their individual effects.

本文提出一种基于视觉的控制新的挑战性基准，用于分析在多种视觉干扰和复杂性环境下的强化学习算法，结果表明当前视觉控制强化学习方法在复杂性视觉干扰下表现不佳，需要新的方法来应对现实世界的视觉复杂性。

分心控制套件 - 基于像素的强化学习的挑战性基准测试

The Distracting Control Suite -- A Challenging Benchmark for  Reinforcement Learning from Pixels

We study active object tracking, where a tracker takes visual observations
(i.e., frame sequences) as input and produces the corresponding camera control
signals as output (e.g., move forward, turn left, etc.). Conventional methods
tackle tracking and camera control tasks separately, and the resulting system
is difficult to tune jointly. These methods also require significant human
efforts for image labeling and expensive trial-and-error system tuning in the
real world. To address these issues, we propose, in this paper, an end-to-end
solution via deep reinforcement learning. A ConvNet-LSTM function approximator
is adopted for the direct frame-to-action prediction. We further propose an
environment augmentation technique and a customized reward function, which are
crucial for successful training. The tracker trained in simulators (ViZDoom and
Unreal Engine) demonstrates good generalization behaviors in the case of unseen
object moving paths, unseen object appearances, unseen backgrounds, and
distracting objects. The system is robust and can restore tracking after
occasional lost of the target being tracked. We also find that the tracking
ability, obtained solely from simulators, can potentially transfer to
real-world scenarios. We demonstrate successful examples of such transfer, via
experiments over the VOT dataset and the deployment of a real-world robot using
the proposed active tracker trained in simulation.

本研究通过深度强化学习解决了目标跟踪和摄像头控制任务难以共同调整的问题，并在模拟器上训练的跟踪器表现出良好的泛化能力，且具备潜在的真实场景应用价值。