Traditional control theory-based methods require tailored engineering for
each system and constant fine-tuning. In power plant control, one often needs
to obtain a precise representation of the system dynamics and carefully design
the control scheme accordingly. Model-free Reinforcement learning (RL) has
emerged as a promising solution for control tasks due to its ability to learn
from trial-and-error interactions with the environment. It eliminates the need
for explicitly modeling the environment's dynamics, which is potentially
inaccurate. However, the direct imposition of state constraints in power plant
control raises challenges for standard RL methods. To address this, we propose
a chance-constrained RL algorithm based on Proximal Policy Optimization for
supervisory control. Our method employs Lagrangian relaxation to convert the
constrained optimization problem into an unconstrained objective, where
trainable Lagrange multipliers enforce the state constraints. Our approach
achieves the smallest distance of violation and violation rate in a load-follow
maneuver for an advanced Nuclear Power Plant design.

我们提出了基于近端策略优化的概率约束强化学习算法，通过使用 Lagrangian relaxation 将约束优化问题转换为无约束目标，从而在先进的核电厂设计中实现了最小违规距离和违规率。