We present PRM-RL, a hierarchical method for long-range navigation task
completion that combines sampling based path planning with reinforcement
learning (RL). The RL agents learn short-range, point-to-point navigation
policies that capture robot dynamics and task constraints without knowledge of
the large-scale topology. Next, the sampling-based planners provide roadmaps
which connect robot configurations that can be successfully navigated by the RL
agent. The same RL agents are used to control the robot under the direction of
the planning, enabling long-range navigation. We use the Probabilistic Roadmaps
(PRMs) for the sampling-based planner. The RL agents are constructed using
feature-based and deep neural net policies in continuous state and action
spaces. We evaluate PRM-RL, both in simulation and on-robot, on two navigation
tasks with non-trivial robot dynamics: end-to-end differential drive indoor
navigation in office environments, and aerial cargo delivery in urban
environments with load displacement constraints. Our results show improvement
in task completion over both RL agents on their own and traditional
sampling-based planners. In the indoor navigation task, PRM-RL successfully
completes up to 215 m long trajectories under noisy sensor conditions, and the
aerial cargo delivery completes flights over 1000 m without violating the task
constraints in an environment 63 million times larger than used in training.

PRM-RL 是一种利用采样路径规划结合强化学习实现远程导航的分层方法，其中 RL 代理通过学习短距离点到点导航策略，采用面向特征和深度神经网络及 PRMs 进行构建，并通过采样计划器提供的规划路径实现机器人的控制。PRM-RL 的应用结果表明，在室内和城市环境下，比起单独使用 RL 代理或传统的采样路径规划方法，PRM-RL 实现了任务完成度的显著提升，在噪声传感器条件下成功完成长达 215 米的轨迹，且实现了对 1,000 米长的空中货运的无违约任务达成。