Energy markets can provide incentives for undesired behavior of market
participants. Multi-agent Reinforcement learning (MARL) is a promising new
approach to determine the expected behavior of energy market participants.
However, reinforcement learning requires many interactions with the system to
converge, and the power system environment often consists of extensive
computations, e.g., optimal power flow (OPF) calculation for market clearing.
To tackle this complexity, we provide a model of the energy market to a basic
MARL algorithm, in form of a learned OPF approximation and explicit market
rules. The learned OPF surrogate model makes an explicit solving of the OPF
completely unnecessary. Our experiments demonstrate that the model additionally
reduces training time by about one order of magnitude, but at the cost of a
slightly worse approximation of the Nash equilibrium. Potential applications of
our method are market design, more realistic modeling of market participants,
and analysis of manipulative behavior.

通过提供一个学习了的 OPF 近似和显式的市场规则的能源市场模型，以 MARL 算法的形式来确定预期的能源市场参与者的行为，从而减少与系统的交互次数，并减少培训时间，同时稍微糟糕地近似了纳什均衡。