This paper proposes a realistic modularized framework for controlling
autonomous surface vehicles (ASVs) on inland waterways (IWs) based on deep
reinforcement learning (DRL). The framework comprises two levels: a high-level
local path planning (LPP) unit and a low-level path following (PF) unit, each
consisting of a DRL agent. The LPP agent is responsible for planning a path
under consideration of nearby vessels, traffic rules, and the geometry of the
waterway. We thereby leverage a recently proposed spatial-temporal recurrent
neural network architecture, which is transferred to continuous action spaces.
The PF agent is responsible for low-level actuator control while accounting for
shallow water influences on the marine craft and the environmental forces
winds, waves, and currents. Both agents are thoroughly validated in simulation,
employing the lower Elbe in northern Germany as an example case and using real
AIS trajectories to model the behavior of other ships.

本文提出了一个基于深度强化学习的逼真的模块化框架，用于控制内陆水路上的自主表面船舶（ASV）。该框架包括两个层次：高级本地路径规划（LPP）单元和低级路径跟踪（PF）单元，每个单元都包含一个 DRL 代理。其中，LPP 代理负责考虑附近船只、交通规则和水路几何形状来规划路径，我们利用了最近提出的空间时间循环神经网络架构，在连续行动空间下进行转移。PF 代理负责低级执行器控制，同时考虑浅水对海船的影响以及环境力量如风、浪和洋流。两个代理在仿真环境中进行了全面验证，以德国北部的下艾尔伯为案例，使用真实的 AIS 轨迹来模拟其他船只的行为。