Autonomous underwater vehicles (AUVs) play a crucial role in surveying marine
environments, carrying out underwater inspection tasks, and ocean exploration.
However, in order to ensure that the AUV is able to carry out its mission
successfully, a control system capable of adapting to changing environmental
conditions is required. Furthermore, to ensure the robotic platform's safe
operation, the onboard controller should be able to operate under certain
constraints. In this work, we investigate the feasibility of Model Predictive
Path Integral Control (MPPI) for the control of an AUV. We utilise a non-linear
model of the AUV to propagate the samples of the MPPI, which allow us to
compute the control action in real time. We provide a detailed evaluation of
the effect of the main hyperparameters on the performance of the MPPI
controller. Furthermore, we compared the performance of the proposed method
with a classical PID and Cascade PID approach, demonstrating the superiority of
our proposed controller. Finally, we present results where environmental
constraints are added and show how MPPI can handle them by simply incorporating
those constraints in the cost function.

自主水下航行器（AUV）在测量海洋环境、进行水下检查任务和海洋探索方面发挥着关键作用。本研究探讨了使用模型预测路径积分控制（MPPI）对 AUV 进行控制的可行性，并对主要超参数对 MPPI 控制器性能的影响进行了详细评估。同时，我们将所提出方法与经典的 PID 和级联 PID 方法进行了比较，证明了我们提出的控制器的优越性。最后，我们加入了环境约束并展示了 MPPI 如何通过将这些约束整合在成本函数中来处理它们。

利用模型预测路径积分控制增强 AUV 自主性

Enhancing AUV Autonomy With Model Predictive Path Integral Control

Maintaining the stability of the modern power grid is becoming increasingly
difficult due to fluctuating power consumption, unstable power supply coming
from renewable energies, and unpredictable accidents such as man-made and
natural disasters. As the operation on the power grid must consider its impact
on future stability, reinforcement learning (RL) has been employed to provide
sequential decision-making in power grid management. However, existing methods
have not considered the environmental constraints. As a result, the learned
policy has risk of selecting actions that violate the constraints in
emergencies, which will escalate the issue of overloaded power lines and lead
to large-scale blackouts. In this work, we propose a novel method for this
problem, which builds on top of the search-based planning algorithm. At the
planning stage, the search space is limited to the action set produced by the
policy. The selected action strictly follows the constraints by testing its
outcome with the simulation function provided by the system. At the learning
stage, to address the problem that gradients cannot be propagated to the
policy, we introduce Evolutionary Strategies (ES) with black-box policy
optimization to improve the policy directly, maximizing the returns of the long
run. In NeurIPS 2020 Learning to Run Power Network (L2RPN) competition, our
solution safely managed the power grid and ranked first in both tracks.

本文提出了一种基于搜索规划算法的新方法解决强化学习中存在的环境约束问题，同时采用黑盒策略优化的进化策略来训练策略直接优化。在 NeurIPS L2RPN 竞赛中，我们的解决方案在两个轨道中均名列第一，能够有效管理电网并确保其地安全性。