Air transportation is undergoing a rapid evolution globally with the
introduction of Advanced Air Mobility (AAM) and with it comes novel challenges
and opportunities for transforming aviation. As AAM operations introduce
increasing heterogeneity in vehicle capabilities and density, increased levels
of automation are likely necessary to achieve operational safety and efficiency
goals. This paper focuses on one example where increased automation has been
suggested. Autonomous operations will need contingency management systems that
can monitor evolving risk across a span of interrelated (or interdependent)
hazards and, if necessary, execute appropriate control interventions via
supervised or automated decision making. Accommodating this complex environment
may require automated functions (autonomy) that apply artificial intelligence
(AI) techniques that can adapt and respond to a quickly changing environment.
This paper explores the use of Deep Reinforcement Learning (DRL) which has
shown promising performance in complex and high-dimensional environments where
the objective can be constructed as a sequential decision-making problem. An
extension of a prior formulation of the contingency management problem as a
Markov Decision Process (MDP) is presented and uses a DRL framework to train
agents that mitigate hazards present in the simulation environment. A
comparison of these learning-based agents and classical techniques is presented
in terms of their performance, verification difficulties, and development
process.

这篇论文探讨了在现代航空运输中引入的先进空中机动性（AAM）所带来的挑战和机遇，着重研究了自主运营需要的控制系统以及使用深度强化学习（DRL）技术进行演示环境中的危害减轻的比较。

高级空中机动性的应急管理中考虑深度强化学习时的权衡

Tradeoffs When Considering Deep Reinforcement Learning for Contingency  Management in Advanced Air Mobility

Advanced Air Mobility (AAM) is the next generation of air transportation that
includes new entrants such as electric vertical takeoff and landing (eVTOL)
aircraft, increasingly autonomous flight operations, and small UAS package
delivery. With these new vehicles and operational concepts comes a desire to
increase densities far beyond what occurs today in and around urban areas, to
utilize new battery technology, and to move toward more autonomously-piloted
aircraft. To achieve these goals, it becomes essential to introduce new safety
management system capabilities that can rapidly assess risk as it evolves
across a span of complex hazards and, if necessary, mitigate risk by executing
appropriate contingencies via supervised or automated decision-making during
flights. Recently, reinforcement learning has shown promise for real-time
decision making across a wide variety of applications including contingency
management. In this work, we formulate the contingency management problem as a
Markov Decision Process (MDP) and integrate the contingency management MDP into
the AAM-Gym simulation framework. This enables rapid prototyping of
reinforcement learning algorithms and evaluation of existing systems, thus
providing a community benchmark for future algorithm development. We report
baseline statistical information for the environment and provide example
performance metrics.

通过整合 AAM-Gym 仿真框架中的紧急情况管理 MDP，本研究将紧急情况管理问题作为 Markov 决策过程，并将其快速原型化为强化学习算法，从而为未来算法开发提供一个社区基准。

面向 AAM 应急管理的标准化强化学习框架

Towards a Standardized Reinforcement Learning Framework for AAM  Contingency Management

Advanced Air Mobility (AAM) introduces a new, efficient mode of
transportation with the use of vehicle autonomy and electrified aircraft to
provide increasingly autonomous transportation between previously underserved
markets. Safe and efficient navigation of low altitude aircraft through highly
dense environments requires the integration of a multitude of complex
observations, such as surveillance, knowledge of vehicle dynamics, and weather.
The processing and reasoning on these observations pose challenges due to the
various sources of uncertainty in the information while ensuring cooperation
with a variable number of aircraft in the airspace. These challenges coupled
with the requirement to make safety-critical decisions in real-time rule out
the use of conventional separation assurance techniques. We present a
decentralized reinforcement learning framework to provide autonomous
self-separation capabilities within AAM corridors with the use of speed and
vertical maneuvers. The problem is formulated as a Markov Decision Process and
solved by developing a novel extension to the sample-efficient, off-policy soft
actor-critic (SAC) algorithm. We introduce the use of attention networks for
variable-length observation processing and a distributed computing architecture
to achieve high training sample throughput as compared to existing approaches.
A comprehensive numerical study shows that the proposed framework can ensure
safe and efficient separation of aircraft in high density, dynamic environments
with various sources of uncertainty.

Advanced Air Mobility 引入了一种新的高效的交通方式，利用车辆自动化和电动飞机在以前服务不足的市场之间提供越来越自主的交通。本文提出了一个基于去中心化强化学习框架的方法，在 AAM 走廊中利用速度和垂直机动提供自主的自主分离能力。通过引入注意力网络用于可变长度的观察处理和分布式计算架构，提高了模型的高训练样本吞吐量，并实现了在具有各种不确定性的高密度、动态环境中确保飞机安全和高效分离的目标。

通过分布式强化学习与注意力网络提高自主分离保障

Improving Autonomous Separation Assurance through Distributed  Reinforcement Learning with Attention Networks

We introduce AAM-Gym, a research and development testbed for Advanced Air
Mobility (AAM). AAM has the potential to revolutionize travel by reducing
ground traffic and emissions by leveraging new types of aircraft such as
electric vertical take-off and landing (eVTOL) aircraft and new advanced
artificial intelligence (AI) algorithms. Validation of AI algorithms require
representative AAM scenarios, as well as a fast time simulation testbed to
evaluate their performance. Until now, there has been no such testbed available
for AAM to enable a common research platform for individuals in government,
industry, or academia. MIT Lincoln Laboratory has developed AAM-Gym to address
this gap by providing an ecosystem to develop, train, and validate new and
established AI algorithms across a wide variety of AAM use-cases. In this
paper, we use AAM-Gym to study the performance of two reinforcement learning
algorithms on an AAM use-case, separation assurance in AAM corridors. The
performance of the two algorithms is demonstrated based on a series of metrics
provided by AAM-Gym, showing the testbed's utility to AAM research.

介绍 AAM-Gym，这是一种研究开发测试平台，用于测试先进空中机动性 (AAM)。使用 AAM-Gym，本文研究了两种强化学习算法在 AAM 走廊分离保障案例中的绩效，并演示了算法的性能。