Dynamic obstacle avoidance (DOA) is a fundamental challenge for any
autonomous vehicle, independent of whether it operates in sea, air, or land.
This paper proposes a two-step architecture for handling DOA tasks by combining
supervised and reinforcement learning (RL). In the first step, we introduce a
data-driven approach to estimate the collision risk of an obstacle using a
recurrent neural network, which is trained in a supervised fashion and offers
robustness to non-linear obstacle movements. In the second step, we include
these collision risk estimates into the observation space of an RL agent to
increase its situational awareness.~We illustrate the power of our two-step
approach by training different RL agents in a challenging environment that
requires to navigate amid multiple obstacles. The non-linear movements of
obstacles are exemplarily modeled based on stochastic processes and periodic
patterns, although our architecture is suitable for any obstacle dynamics. The
experiments reveal that integrating our collision risk metrics into the
observation space doubles the performance in terms of reward, which is
equivalent to halving the number of collisions in the considered environment.
Furthermore, we show that the architecture's performance improvement is
independent of the applied RL algorithm.

本文提出了一种两步架构，通过结合监督学习和强化学习来处理动态避障任务。第一步，我们采用数据驱动方法使用循环神经网络估计障碍物的碰撞风险，以解决非线性障碍物移动的鲁棒性问题。第二步，我们将这些碰撞风险估计值纳入强化学习智能体的观察空间，以增加其环境感知能力。我们通过在具有多个障碍物的复杂环境中训练不同的强化学习智能体来展示我们的两步方法的有效性。实验证明，将碰撞风险指标整合到观察空间中，可以使智能体的奖励性能提升一倍，相当于在考虑的环境中减少一半的碰撞次数。此外，我们还证明该架构的性能改进与所采用的强化学习算法无关。

两步式动态避障

Two-step dynamic obstacle avoidance

Multi-agent reinforcement learning based methods are significant for online
planning of feasible and safe paths for agents in dynamic and uncertain
scenarios. Although some methods like fully centralized and fully decentralized
methods achieve a certain measure of success, they also encounter problems such
as dimension explosion and poor convergence, respectively. In this paper, we
propose a novel centralized training with decentralized execution method based
on multi-agent reinforcement learning to solve the dynamic obstacle avoidance
problem online. In this approach, each agent communicates only with the central
planner or only with its neighbors, respectively, to plan feasible and safe
paths online. We improve our methods based on the idea of model predictive
control to increase the training efficiency and sample utilization of agents.
The experimental results in both simulation, indoor, and outdoor environments
validate the effectiveness of our method. The video is available at
this https URL

我们提出了一种基于多智能体强化学习的新颖集中训练与分散执行方法，用于在线解决动态避障问题。改进方法采用模型预测控制的思想提高智能体的训练效率和样本利用率，并通过模拟、室内和室外环境的实验结果验证了方法的有效性。

多智能体强化学习下的动态避障无人机路径规划

UAV Pathfinding in Dynamic Obstacle Avoidance with Multi-agent  Reinforcement Learning

Dynamic obstacle avoidance on quadrotors requires low latency. A class of
sensors that are particularly suitable for such scenarios are event cameras. In
this paper, we present a deep learning -- based solution for dodging multiple
dynamic obstacles on a quadrotor with a single event camera and on-board
computation. Our approach uses a series of shallow neural networks for
estimating both the ego-motion and the motion of independently moving objects.
The networks are trained in simulation and directly transfer to the real world
without any fine-tuning or retraining. We successfully evaluate and demonstrate
the proposed approach in many real-world experiments with obstacles of
different shapes and sizes, achieving an overall success rate of 70% including
objects of unknown shape and a low light testing scenario. To our knowledge,
this is the first deep learning -- based solution to the problem of dynamic
obstacle avoidance using event cameras on a quadrotor. Finally, we also extend
our work to the pursuit task by merely reversing the control policy, proving
that our navigation stack can cater to different scenarios.

本文提出基于深度学习的方法利用事件相机和机载计算实现四轴飞行器避开多个动态障碍物。该方法通过一系列浅层神经网络模型同时估计自我运动和独立运动目标的运动，并在真实场景下实现测试并证明有效性，成功率达到 70%。此外，本方法通过反向控制来延伸应用于其他不同场景，是首次使用事件相机实现基于深度学习的动态障碍物避免方案的研究。