Robot navigation using deep reinforcement learning (DRL) has shown great
potential in improving the performance of mobile robots. Nevertheless, most
existing DRL-based navigation methods primarily focus on training a policy that
directly commands the robot with low-level controls, like linear and angular
velocities, which leads to unstable speeds and unsmooth trajectories of the
robot during the long-term execution. An alternative method is to train a DRL
policy that outputs the navigation path directly. However, two roadblocks arise
for training a DRL policy that outputs paths: (1) The action space for
potential paths often involves higher dimensions comparing to low-level
commands, which increases the difficulties of training; (2) It takes multiple
time steps to track a path instead of a single time step, which requires the
path to predicate the interactions of the robot w.r.t. the dynamic environment
in multiple time steps. This, in turn, amplifies the challenges associated with
training. In response to these challenges, we propose PathRL, a novel DRL
method that trains the policy to generate the navigation path for the robot.
Specifically, we employ specific action space discretization techniques and
tailored state space representation methods to address the associated
challenges. In our experiments, PathRL achieves better success rates and
reduces angular rotation variability compared to other DRL navigation methods,
facilitating stable and smooth robot movement. We demonstrate the competitive
edge of PathRL in both real-world scenarios and multiple challenging simulation
environments.

通过使用深度强化学习 (DRL) 的机器人导航，可以提高移动机器人的性能。然而，现有的基于 DRL 的导航方法主要集中于训练一个直接命令机器人进行低级控制的策略，导致机器人在长期执行过程中速度不稳定、轨迹不平滑。我们提出了一种名为 PathRL 的新型 DRL 方法，通过训练策略来生成机器人的导航路径，具有更高的成功率和减少了角度旋转变异性的优势，实现了稳定和流畅的机器人移动。