Recent challenges in operating power networks arise from increasing energy
demands and unpredictable renewable sources like wind and solar. While
reinforcement learning (RL) shows promise in managing these networks, through
topological actions like bus and line switching, efficiently handling large
action spaces as networks grow is crucial. This paper presents a hierarchical
multi-agent reinforcement learning (MARL) framework tailored for these
expansive action spaces, leveraging the power grid's inherent hierarchical
nature. Experimental results indicate the MARL framework's competitive
performance with single-agent RL methods. We also compare different RL
algorithms for lower-level agents alongside different policies for higher-order
agents.

该论文提出了一种适用于扩展动作空间的分层多智能体强化学习 (MARL) 框架，利用电网固有的分层结构，用于管理大规模的电力网络，并且实验结果表明该框架的性能与单智能体强化学习方法相当。

电力系统拓扑优化的多智能体强化学习

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

The application of artificial intelligence to simulate air-to-air combat
scenarios is attracting increasing attention. To date the high-dimensional
state and action spaces, the high complexity of situation information (such as
imperfect and filtered information, stochasticity, incomplete knowledge about
mission targets) and the nonlinear flight dynamics pose significant challenges
for accurate air combat decision-making. These challenges are exacerbated when
multiple heterogeneous agents are involved. We propose a hierarchical
multi-agent reinforcement learning framework for air-to-air combat with
multiple heterogeneous agents. In our framework, the decision-making process is
divided into two stages of abstraction, where heterogeneous low-level policies
control the action of individual units, and a high-level commander policy
issues macro commands given the overall mission targets. Low-level policies are
trained for accurate unit combat control. Their training is organized in a
learning curriculum with increasingly complex training scenarios and
league-based self-play. The commander policy is trained on mission targets
given pre-trained low-level policies. The empirical validation advocates the
advantages of our design choices.

应用人工智能模拟空战场景吸引了越来越多的关注，本文提出了一种用于多个异构代理的空中对空战斗的层次化多代理强化学习框架，其中命令决策过程划分为两个抽象阶段，低级政策控制个体单位的行动，高级指挥官政策根据总体任务目标下达宏观指令，并通过针对预先训练的低级政策对任务目标进行训练，实验证明了我们设计的优点。