Sequential Decision Making under Uncertainty (SDMU) is ubiquitous in many
domains such as energy, finance, and supply chains. Some SDMU applications are
naturally modeled as Multistage Stochastic Optimization Problems (MSPs), but
the resulting optimizations are notoriously challenging from a computational
standpoint. Under assumptions of convexity and stage-wise independence of the
uncertainty, the resulting optimization can be solved efficiently using
Stochastic Dual Dynamic Programming (SDDP). Two-stage Linear Decision Rules
(TS-LDRs) have been proposed to solve MSPs without the stage-wise independence
assumption. TS-LDRs are computationally tractable, but using a policy that is a
linear function of past observations is typically not suitable for non-convex
environments arising, for example, in energy systems. This paper introduces a
novel approach, Two-Stage General Decision Rules (TS-GDR), to generalize the
policy space beyond linear functions, making them suitable for non-convex
environments. TS-GDR is a self-supervised learning algorithm that trains the
nonlinear decision rules using stochastic gradient descent (SGD); its forward
passes solve the policy implementation optimization problems, and the backward
passes leverage duality theory to obtain closed-form gradients. The
effectiveness of TS-GDR is demonstrated through an instantiation using Deep
Recurrent Neural Networks named Two-Stage Deep Decision Rules (TS-DDR). The
method inherits the flexibility and computational performance of Deep Learning
methodologies to solve SDMU problems generally tackled through large-scale
optimization techniques. Applied to the Long-Term Hydrothermal Dispatch (LTHD)
problem using actual power system data from Bolivia, the TS-DDR not only
enhances solution quality but also significantly reduces computation times by
several orders of magnitude.

在这篇论文中，介绍了一种名为 Two-Stage General Decision Rules (TS-GDR) 的新方法，通过使用深度递归神经网络来解决顺序决策问题，并应用于长期水热调度问题，显著提高了解决方案质量并大幅减少计算时间。

两阶段 ML 引导的不确定性顺序决策规则

Two-Stage ML-Guided Decision Rules for Sequential Decision Making under  Uncertainty

Solving large-scale multistage stochastic programming (MSP) problems poses a
significant challenge as commonly used stagewise decomposition algorithms,
including stochastic dual dynamic programming (SDDP), face growing time
complexity as the subproblem size and problem count increase. Traditional
approaches approximate the value functions as piecewise linear convex functions
by incrementally accumulating subgradient cutting planes from the primal and
dual solutions of stagewise subproblems. Recognizing these limitations, we
introduce TranSDDP, a novel Transformer-based stagewise decomposition
algorithm. This innovative approach leverages the structural advantages of the
Transformer model, implementing a sequential method for integrating subgradient
cutting planes to approximate the value function. Through our numerical
experiments, we affirm TranSDDP's effectiveness in addressing MSP problems. It
efficiently generates a piecewise linear approximation for the value function,
significantly reducing computation time while preserving solution quality, thus
marking a promising progression in the treatment of large-scale multistage
stochastic programming problems.

通过引入 TranSDDP，一种基于 Transformer 的分阶段分解算法，我们有效地为大规模多阶段随机规划问题生成了分段线性逼近，显著减少了计算时间同时保持解决方案的质量，从而标志着在处理大规模多阶段随机规划问题方面的一个有希望的进展。