Dimensionality reduction is crucial for controlling nonlinear partial
differential equations (PDE) through a "reduce-then-design" strategy, which
identifies a reduced-order model and then implements model-based control
solutions. However, inaccuracies in the reduced-order modeling can
substantially degrade controller performance, especially in PDEs with chaotic
behavior. To address this issue, we augment the reduce-then-design procedure
with a policy optimization (PO) step. The PO step fine-tunes the model-based
controller to compensate for the modeling error from dimensionality reduction.
This augmentation shifts the overall strategy into
reduce-then-design-then-adapt, where the model-based controller serves as a
warm start for PO. Specifically, we study the state-feedback tracking control
of PDEs that aims to align the PDE state with a specific constant target
subject to a linear-quadratic cost. Through extensive experiments, we show that
a few iterations of PO can significantly improve the model-based controller
performance. Our approach offers a cost-effective alternative to PDE control
using end-to-end reinforcement learning.

通过增加策略优化步骤，将模型控制器与降维模型相结合，实现对混沌行为的偏差建模误差补偿，提高模型控制器性能。