Deep reinforcement learning (DRL) for fluidic pinball, three individually
rotating cylinders in the uniform flow arranged in an equilaterally triangular
configuration, can learn the efficient flow control strategies due to the
validity of self-learning and data-driven state estimation for complex fluid
dynamic problems. In this work, we present a DRL-based real-time feedback
strategy to control the hydrodynamic force on fluidic pinball, i.e., force
extremum and tracking, from cylinders' rotation. By adequately designing reward
functions and encoding historical observations, and after automatic learning of
thousands of iterations, the DRL-based control was shown to make reasonable and
valid control decisions in nonparametric control parameter space, which is
comparable to and even better than the optimal policy found through lengthy
brute-force searching. Subsequently, one of these results was analyzed by a
machine learning model that enabled us to shed light on the basis of
decision-making and physical mechanisms of the force tracking process. The
finding from this work can control hydrodynamic force on the operation of
fluidic pinball system and potentially pave the way for exploring efficient
active flow control strategies in other complex fluid dynamic problems.

本文使用深度强化学习算法构建了一个实时反馈策略，用于实现流体弹球系统的流体动力学力控制和追踪，结果表明该策略在复杂流体动力学问题上的控制决策效果理想，可为其他复杂流体动力学问题的高效活动控制策略提供探索。