The optimal dispatch of energy storage systems (ESSs) presents formidable
challenges due to the uncertainty introduced by fluctuations in dynamic prices,
demand consumption, and renewable-based energy generation. By exploiting the
generalization capabilities of deep neural networks (DNNs), deep reinforcement
learning (DRL) algorithms can learn good-quality control models that adaptively
respond to distribution networks' stochastic nature. However, current DRL
algorithms lack the capabilities to enforce operational constraints strictly,
often even providing unfeasible control actions. To address this issue, we
propose a DRL framework that effectively handles continuous action spaces while
strictly enforcing the environments and action space operational constraints
during online operation. Firstly, the proposed framework trains an action-value
function modeled using DNNs. Subsequently, this action-value function is
formulated as a mixed-integer programming (MIP) formulation enabling the
consideration of the environment's operational constraints. Comprehensive
numerical simulations show the superior performance of the proposed MIP-DRL
framework, effectively enforcing all constraints while delivering high-quality
dispatch decisions when compared with state-of-the-art DRL algorithms and the
optimal solution obtained with a perfect forecast of the stochastic variables.

提出了一种利用深度强化学习和混合整数规划的框架，用于优化能量储存系统的调度决策，同时严格执行操作约束，以及在存在不确定性的环境中取得高质量的调度决策。