End-to-end deep reinforcement learning (DRL) for quadrotor control promises
many benefits -- easy deployment, task generalization and real-time execution
capability. Prior end-to-end DRL-based methods have showcased the ability to
deploy learned controllers onto single quadrotors or quadrotor teams
maneuvering in simple, obstacle-free environments. However, the addition of
obstacles increases the number of possible interactions exponentially, thereby
increasing the difficulty of training RL policies. In this work, we propose an
end-to-end DRL approach to control quadrotor swarms in environments with
obstacles. We provide our agents a curriculum and a replay buffer of the
clipped collision episodes to improve performance in obstacle-rich
environments. We implement an attention mechanism to attend to the neighbor
robots and obstacle interactions - the first successful demonstration of this
mechanism on policies for swarm behavior deployed on severely
compute-constrained hardware. Our work is the first work that demonstrates the
possibility of learning neighbor-avoiding and obstacle-avoiding control
policies trained with end-to-end DRL that transfers zero-shot to real
quadrotors. Our approach scales to 32 robots with 80% obstacle density in
simulation and 8 robots with 20% obstacle density in physical deployment. Video
demonstrations are available on the project website at:
this https URL

在包含障碍物的环境中，我们提出一种使用端到端深度强化学习（DRL）控制无人机群的方法，该方法通过提供课程和剪辑碰撞情景的重放缓冲区来改善性能，并且实现了对邻居机器人和障碍物交互的注意机制。这是首次成功地将该机制应用于在计算资源受限的硬件上部署的群体行为策略，我们的方法可以在模拟环境中实现 32 架机器人并达到 80％的障碍物密度，在实际物理环境中可以实现 8 架机器人并达到 20％的障碍物密度，并且可以零 - shot 地转移到真实的无人机上。

基于端到端强化学习的四旋翼群体的避障和导航

Collision Avoidance and Navigation for a Quadrotor Swarm Using  End-to-end Deep Reinforcement Learning

Swarm behavior emerges from the local interaction of agents and their
environment often encoded as simple rules. Extracting the rules by watching a
video of the overall swarm behavior could help us study and control swarm
behavior in nature, or artificial swarms that have been designed by external
actors. It could also serve as a new source of inspiration for swarm robotics.
Yet extracting such rules is challenging as there is often no visible link
between the emergent properties of the swarm and their local interactions. To
this end, we develop a method to automatically extract understandable swarm
controllers from video demonstrations. The method uses evolutionary algorithms
driven by a fitness function that compares eight high-level swarm metrics. The
method is able to extract many controllers (behavior trees) in a simple
collective movement task. We then provide a qualitative analysis of behaviors
that resulted in different trees, but similar behaviors. This provides the
first steps toward automatic extraction of swarm controllers based on
observations.

我们开发了一种方法，可以从视频演示中自动提取可理解的群体控制器，该方法使用进化算法根据比较高级别群体度量值的适应函数提取许多控制器，在收集运动任务中，并提供不同树导致的不同行为的定性分析，这是实现基于观察的群控器自动提取的第一步。