Active Voltage Control (AVC) on the Power Distribution Networks (PDNs) aims
to stabilize the voltage levels to ensure efficient and reliable operation of
power systems. With the increasing integration of distributed energy resources,
recent efforts have explored employing multi-agent reinforcement learning
(MARL) techniques to realize effective AVC. Existing methods mainly focus on
the acquisition of short-term AVC strategies, i.e., only learning AVC within
the short-term training trajectories of a singular diurnal cycle. However, due
to the dynamic nature of load demands and renewable energy, the operation
states of real-world PDNs may exhibit significant distribution shifts across
varying timescales (e.g., daily and seasonal changes). This can render those
short-term strategies suboptimal or even obsolete when performing continuous
AVC over extended periods. In this paper, we propose a novel temporal
prototype-aware learning method, abbreviated as TPA, to learn time-adaptive AVC
under short-term training trajectories. At the heart of TPA are two
complementary components, namely multi-scale dynamic encoder and temporal
prototype-aware policy, that can be readily incorporated into various MARL
methods. The former component integrates a stacked transformer network to learn
underlying temporal dependencies at different timescales of the PDNs, while the
latter implements a learnable prototype matching mechanism to construct a
dedicated AVC policy that can dynamically adapt to the evolving operation
states. Experimental results on the AVC benchmark with different PDN sizes
demonstrate that the proposed TPA surpasses the state-of-the-art counterparts
not only in terms of control performance but also by offering model
transferability. Our code is available at
this https URL

提出了一种新的时间原型感知学习方法来学习短期训练轨迹下的时变电压控制，该方法在控制性能上不仅超过了现有技术，还提供了模型可转移性。

基于时间原型感知的主动电压控制在电力配电网络上的学习

Temporal Prototype-Aware Learning for Active Voltage Control on Power  Distribution Networks

Active voltage control presents a promising avenue for relieving power
congestion and enhancing voltage quality, taking advantage of the distributed
controllable generators in the power network, such as roof-top photovoltaics.
While Multi-Agent Reinforcement Learning (MARL) has emerged as a compelling
approach to address this challenge, existing MARL approaches tend to overlook
the constrained optimization nature of this problem, failing in guaranteeing
safety constraints. In this paper, we formalize the active voltage control
problem as a constrained Markov game and propose a safety-constrained MARL
algorithm. We expand the primal-dual optimization RL method to multi-agent
settings, and augment it with a novel approach of double safety estimation to
learn the policy and to update the Lagrange-multiplier. In addition, we
proposed different cost functions and investigated their influences on the
behavior of our constrained MARL method. We evaluate our approach in the power
distribution network simulation environment with real-world scale scenarios.
Experimental results demonstrate the effectiveness of the proposed method
compared with the state-of-the-art MARL methods.

基于多智能体强化学习的安全限制算法，用于解决分布式可控发电机在电力网络中主动电压控制问题，通过在电力分布网络模拟环境中进行实验，证明了该方法与最新的多智能体强化学习方法相比的有效性。

安全约束的多智能体强化学习在主动电压控制中的应用

Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage  Control

This paper presents a problem in power networks that creates an exciting and
yet challenging real-world scenario for application of multi-agent
reinforcement learning (MARL). The emerging trend of decarbonisation is placing
excessive stress on power distribution networks. Active voltage control is seen
as a promising solution to relieve power congestion and improve voltage quality
without extra hardware investment, taking advantage of the controllable
apparatuses in the network, such as roof-top photovoltaics (PVs) and static var
compensators (SVCs). These controllable apparatuses appear in a vast number and
are distributed in a wide geographic area, making MARL a natural candidate.
This paper formulates the active voltage control problem in the framework of
Dec-POMDP and establishes an open-source environment. It aims to bridge the gap
between the power community and the MARL community and be a drive force towards
real-world applications of MARL algorithms. Finally, we analyse the special
characteristics of the active voltage control problems that cause challenges
(e.g. interpretability) for state-of-the-art MARL approaches, and summarise the
potential directions.

本研究使用多智能体强化学习模型解决由去碳化趋势造成的电力网络中的电压拥塞问题，将主要问题转化为 Dec-POMDP 模型并建立开源环境，解决多智能体强化学习在状态可解释性等方面的挑战，为电力社区和多智能体强化学习社区寻求现实应用提供了实践基础和方向参考。