Learning-based methods, particularly Reinforcement Learning (RL), hold great
promise for streamlining deployment, enhancing performance, and achieving
generalization in the control of autonomous multirotor aerial vehicles. Deep RL
has been able to control complex systems with impressive fidelity and agility
in simulation but the simulation-to-reality transfer often brings a
hard-to-bridge reality gap. Moreover, RL is commonly plagued by prohibitively
long training times. In this work, we propose a novel asymmetric
actor-critic-based architecture coupled with a highly reliable RL-based
training paradigm for end-to-end quadrotor control. We show how curriculum
learning and a highly optimized simulator enhance sample complexity and lead to
fast training times. To precisely discuss the challenges related to
low-level/end-to-end multirotor control, we also introduce a taxonomy that
classifies the existing levels of control abstractions as well as
non-linearities and domain parameters. Our framework enables
Simulation-to-Reality (Sim2Real) transfer for direct RPM control after only 18
seconds of training on a consumer-grade laptop as well as its deployment on
microcontrollers to control a multirotor under real-time guarantees. Finally,
our solution exhibits competitive performance in trajectory tracking, as
demonstrated through various experimental comparisons with existing
state-of-the-art control solutions using a real Crazyflie nano quadrotor. We
open source the code including a very fast multirotor dynamics simulator that
can simulate about 5 months of flight per second on a laptop GPU. The fast
training times and deployment to a cheap, off-the-shelf quadrotor lower the
barriers to entry and help democratize the research and development of these
systems.

通过使用基于强化学习的异态 actor-critic 架构和高度可靠的基于强化学习的训练方法，本研究提出了一种新颖的无人机控制框架，能够在仅 18 秒的训练时间内实现从模拟环境到真实环境的快速转化，并能在廉价的、现成的无人机上实现实时控制。同时，本研究通过介绍控制抽象、非线性和领域参数的分类以及引入一个全新的课程学习和高度优化的模拟器，提高了样本的复杂度和训练时间，并通过与现有控制解决方案的实验比较，展示了该框架在航迹跟踪方面具有竞争性能。