We present the PowerGridworld software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld is an open-source software package that helps to fill this gap. To highlight PowerGridworld's key features, we present two case studies and demonstrate learning MARL policies using both OpenAI's multi-agent deep deterministic policy gradient (MADDPG) and RLLib's proximal policy optimization (PPO) algorithms. In both cases, at least some subset of agents incorporates elements of the power flow solution at each time step as part of their reward (negative cost) structures.

本研究介绍了PowerGridworld软件包，它是一个轻量级、模块化、可定制的框架，用于创建面向电力系统的多智能体Gym环境，并可与现有的强化学习训练框架集成。通过两个案例研究，证明了PowerGridworld可以快速实现多智能体RL策略的学习，并支持多设备、组合式的电力系统。

PowerGridworld: 电力系统中多智能体强化学习框架