This paper presents a novel safe reinforcement learning algorithm for strategic bidding of Virtual Power Plants (VPPs) in day-ahead electricity markets. The proposed algorithm utilizes the Deep Deterministic Policy Gradient (DDPG) method to learn competitive bidding policies without requiring an accurate market model. Furthermore, to account for the complex internal physical constraints of VPPs we introduce two enhancements to the DDPG method. Firstly, a projection-based safety shield that restricts the agent's actions to the feasible space defined by the non-linear power flow equations and operating constraints of distributed energy resources is derived. Secondly, a penalty for the shield activation in the reward function that incentivizes the agent to learn a safer policy is introduced. A case study based on the IEEE 13-bus network demonstrates the effectiveness of the proposed approach in enabling the agent to learn a highly competitive, safe strategic policy.

该论文提出了一种新颖的安全强化学习算法，用于虚拟电力厂在日前电力市场上的战略竞标，同时考虑了虚拟电力厂内部复杂的物理约束，并结合DDPG方法设计了投影安全屏障和奖励机制，最终在IEEE 13-bus网络上展示了提出方法的有效性。

面向日前市场的虚拟电力厂战略出价安全强化学习