The Space-Air-Ground Integrated Network (SAGIN), integrating heterogeneous
devices including low earth orbit (LEO) satellites, unmanned aerial vehicles
(UAVs), and ground users (GUs), holds significant promise for advancing smart
city applications. However, resource management of the SAGIN is a challenge
requiring urgent study in that inappropriate resource management will cause
poor data transmission, and hence affect the services in smart cities. In this
paper, we develop a comprehensive SAGIN system that encompasses five distinct
communication links and propose an efficient cooperative multi-type multi-agent
deep reinforcement learning (CMT-MARL) method to address the resource
management issue. The experimental results highlight the efficacy of the
proposed CMT-MARL, as evidenced by key performance indicators such as the
overall transmission rate and transmission success rate. These results
underscore the potential value and feasibility of future implementation of the
SAGIN.

将低地球轨道卫星、无人机和地面用户融合的空地一体化网络（SAGIN），在智慧城市应用方面具有巨大潜力。然而，SAGIN 的资源管理是一个挑战，需要紧急研究，因为不适当的资源管理会导致数据传输差，从而影响智慧城市的服务。本文开发了一个综合的 SAGIN 系统，包括五个不同的通信链路，并提出了一种高效的合作多类型多智能体深度强化学习（CMT-MARL）方法来解决资源管理问题。实验结果突出了所提出的 CMT-MARL 的有效性，体现在整体传输率和传输成功率等关键性能指标上。这些结果证明了未来实施 SAGIN 的潜在价值和可行性。

空中 - 地面 - 空中集成网络资源管理的合作多类型多智能体深度强化学习

Cooperative Multi-Type Multi-Agent Deep Reinforcement Learning for  Resource Management in Space-Air-Ground Integrated Networks

Traditional ground wireless communication networks cannot provide
high-quality services for artificial intelligence (AI) applications such as
intelligent transportation systems (ITS) due to deployment, coverage and
capacity issues. The space-air-ground integrated network (SAGIN) has become a
research focus in the industry. Compared with traditional wireless
communication networks, SAGIN is more flexible and reliable, and it has wider
coverage and higher quality of seamless connection. However, due to its
inherent heterogeneity, time-varying and self-organizing characteristics, the
deployment and use of SAGIN still faces huge challenges, among which the
orchestration of heterogeneous resources is a key issue. Based on virtual
network architecture and deep reinforcement learning (DRL), we model SAGIN's
heterogeneous resource orchestration as a multi-domain virtual network
embedding (VNE) problem, and propose a SAGIN cross-domain VNE algorithm. We
model the different network segments of SAGIN, and set the network attributes
according to the actual situation of SAGIN and user needs. In DRL, the agent is
acted by a five-layer policy network. We build a feature matrix based on
network attributes extracted from SAGIN and use it as the agent training
environment. Through training, the probability of each underlying node being
embedded can be derived. In test phase, we complete the embedding process of
virtual nodes and links in turn based on this probability. Finally, we verify
the effectiveness of the algorithm from both training and testing.

通过使用虚拟网络架构和深度强化学习，提出了一种针对空天地一体化网络（SAGIN）的跨域虚拟网络嵌入（VNE）算法，以解决其具有异构性和动态性的部署和使用面临的挑战，该算法能够从 SAGIN 中提取网络属性并训练智能代理从而有效完成虚拟节点和虚拟链路的嵌入，以应用于智能交通系统等人工智能应用。

基于虚拟网络架构的空中地面一体化多域网络资源编排：一种 DRL 方法

Space-Air-Ground Integrated Multi-domain Network Resource Orchestration based on Virtual Network Architecture: a DRL Method

In this paper, we investigate a computing task scheduling problem in
space-air-ground integrated network (SAGIN) for delay-oriented Internet of
Things (IoT) services. In the considered scenario, an unmanned aerial vehicle
(UAV) collects computing tasks from IoT devices and then makes online
offloading decisions, in which the tasks can be processed at the UAV or
offloaded to the nearby base station or the remote satellite. Our objective is
to design a task scheduling policy that minimizes offloading and computing
delay of all tasks given the UAV energy capacity constraint. To this end, we
first formulate the online scheduling problem as an energy-constrained Markov
decision process (MDP). Then, considering the task arrival dynamics, we develop
a novel deep risk-sensitive reinforcement learning algorithm. Specifically, the
algorithm evaluates the risk, which measures the energy consumption that
exceeds the constraint, for each state and searches the optimal parameter
weighing the minimization of delay and risk while learning the optimal policy.
Extensive simulation results demonstrate that the proposed algorithm can reduce
the task processing delay by up to 30% compared to probabilistic configuration
methods while satisfying the UAV energy capacity constraint.

本文旨在设计一种任务调度策略，以最小化所有任务的离线和计算延迟，同时满足无人机能源容量约束下的延迟导向物联网服务需求，并考虑到任务到达动态变化的情况，提出了基于风险敏感的强化学习算法来解决能耗风险约束下的决策问题。