To solve the optimal power flow (OPF) problem, reinforcement learning (RL) emerges as a promising new approach. However, the RL-OPF literature is strongly divided regarding the exact formulation of the OPF problem as an RL environment. In this work, we collect and implement diverse environment design decisions from the literature regarding training data, observation space, episode definition, and reward function choice. In an experimental analysis, we show the significant impact of these environment design options on RL-OPF training performance. Further, we derive some first recommendations regarding the choice of these design decisions. The created environment framework is fully open-source and can serve as a benchmark for future research in the RL-OPF field.

解决最优功率流问题的强化学习在开放式电力系统建模与优化问题领域显示出了巨大的潜力，然而，如何准确地将最优功率流问题构建为一个强化学习环境的问题，该领域的研究存在明显的分歧。本文通过收集和实现关于训练数据、观测空间、回合定义以及奖励函数选择的多样化环境设计策略来探究这一问题，并通过实验分析展示了这些环境设计选项对强化学习在最优功率流问题上的训练性能的显著影响，为解决这些设计选择问题提出了一些建议，同时开放了用于强化学习在最优功率流问题上的环境框架作为未来研究的基准。

学习最优潮流：环境设计的重要性