Learning-based adaptive control methods hold the premise of enabling
autonomous agents to reduce the effect of process variations with minimal human
intervention. However, its application to autonomous underwater vehicles (AUVs)
has so far been restricted due to 1) unknown dynamics under the form of sea
current disturbance that we can not model properly nor measure due to limited
sensor capability and 2) the nonlinearity of AUVs tasks where the controller
response at some operating points must be overly conservative in order to
satisfy the specification at other operating points. Deep Reinforcement
Learning (DRL) can alleviates these limitations by training general-purpose
neural network policies, but applications of DRL algorithms to AUVs have been
restricted to simulated environments, due to their inherent high sample
complexity and distribution shift problem. This paper presents a novel
approach, merging the Maximum Entropy Deep Reinforcement Learning framework
with a classic model-based control architecture, to formulate an adaptive
controller. Within this framework, we introduce a Sim-to-Real transfer strategy
comprising the following components: a bio-inspired experience replay
mechanism, an enhanced domain randomisation technique, and an evaluation
protocol executed on a physical platform. Our experimental assessments
demonstrate that this method effectively learns proficient policies from
suboptimal simulated models of the AUV, resulting in control performance 3
times higher when transferred to a real-world vehicle, compared to its
model-based nonadaptive but optimal counterpart.

利用学习驱动的自适应控制方法，结合基于模型的控制架构以及最大熵深度强化学习框架，提出了一种适应控制器的创新方法，通过生物启发的经验回放机制、增强的领域随机化技术和在物理平台上的评估协议来实现仿真到真实的迁移策略，实验证明该方法能够从 AUV 的次优仿真模型中学到有效的策略，并在实际环境中将控制性能提高 3 倍。