Previous works showed that Deep-RL can be applied to perform mapless
navigation, including the medium transition of Hybrid Unmanned Aerial
Underwater Vehicles (HUAUVs). This paper presents new approaches based on the
state-of-the-art actor-critic algorithms to address the navigation and medium
transition problems for a HUAUV. We show that a double critic Deep-RL with
Recurrent Neural Networks improves the navigation performance of HUAUVs using
solely range data and relative localization. Our Deep-RL approaches achieved
better navigation and transitioning capabilities with a solid generalization of
learning through distinct simulated scenarios, outperforming previous
approaches.

本文介绍了一种基于最新的 actor-critic 算法的新方法，以解决 HUAUV 的导航和介质转换问题。我们展示了一种双重评论家 Deep-RL 的方法，通过仅使用范围数据和相对定位来提高 HUAUV 的导航性能。我们的 Deep-RL 方法在不同模拟场景中实现了学习的坚实泛化，表现出更好的导航和转换能力，优于以前的方法。

利用深度强化学习进行无地图水空混合飞行器导航与环境通用化

Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Deep Reinforcement Learning Through Environmental Generalization

We present a modern scalable reinforcement learning agent called SEED
(Scalable, Efficient Deep-RL). By effectively utilizing modern accelerators, we
show that it is not only possible to train on millions of frames per second but
also to lower the cost of experiments compared to current methods. We achieve
this with a simple architecture that features centralized inference and an
optimized communication layer. SEED adopts two state of the art distributed
algorithms, IMPALA/V-trace (policy gradients) and R2D2 (Q-learning), and is
evaluated on Atari-57, DeepMind Lab and Google Research Football. We improve
the state of the art on Football and are able to reach state of the art on
Atari-57 three times faster in wall-time. For the scenarios we consider, a 40%
to 80% cost reduction for running experiments is achieved. The implementation
along with experiments is open-sourced so results can be reproduced and novel
ideas tried out.

SEED 是一种现代可扩展强化学习代理程序，采用分布式算法 IMPALA/V-trace 和 R2D2，可以在 Atari-57、DeepMind Lab 和 Google Research Football 上进行评估，并且比当前方法更具成本效益。

SEED RL：可伸缩高效的深度强化学习与加速化的中心推理

SEED RL: Scalable and Efficient Deep-RL with Accelerated Central  Inference

Insertion is a challenging haptic and visual control problem with significant
practical value for manufacturing. Existing approaches in the model-based
robotics community can be highly effective when task geometry is known, but are
complex and cumbersome to implement, and must be tailored to each individual
problem by a qualified engineer. Within the learning community there is a long
history of insertion research, but existing approaches are typically either too
sample-inefficient to run on real robots, or assume access to high-level object
features, e.g. socket pose. In this paper we show that relatively minor
modifications to an off-the-shelf Deep-RL algorithm (DDPG), combined with a
small number of human demonstrations, allows the robot to quickly learn to
solve these tasks efficiently and robustly. Our approach requires no modeling
or simulation, no parameterized search or alignment behaviors, no vision system
aside from raw images, and no reward shaping. We evaluate our approach on a
narrow-clearance peg-insertion task and a deformable clip-insertion task, both
of which include variability in the socket position. Our results show that
these tasks can be solved reliably on the real robot in less than 10 minutes of
interaction time, and that the resulting policies are robust to variance in the
socket position and orientation.

利用深度强化学习，在少量人类演示的情况下，无需模拟和建模就能达到高效且稳健地解决插入问题的效果，且在插入位置具有差异的情况下依然有效。